Wait for enough elements to load in flaky e2e tests#4193
Merged
AnOctopus merged 1 commit intostreamlit:developfrom Dec 17, 2021
Merged
Wait for enough elements to load in flaky e2e tests#4193AnOctopus merged 1 commit intostreamlit:developfrom
AnOctopus merged 1 commit intostreamlit:developfrom
Conversation
The core problem is described in https://docs.cypress.io/guides/core-concepts/retry-ability#Only-the-last-command-is-retried Chaining `get` and then immediately `eq` is prone to flakiness, because if the get call sees some but not enough elements, the eq call will fail, but only the eq call will be retried, so it'll continue to fail even when the actual page loads the elements before the eq call times out. Adding an assertion that there are enough elements ensures that the initial query will be retried until enough elements are present.
kmcgrady
approved these changes
Dec 17, 2021
Collaborator
kmcgrady
left a comment
kmcgrady
left a comment
There was a problem hiding this comment.
Hoping for less flakiness! Thanks for tackling this!
tconkling
added a commit
to tconkling/streamlit
that referenced
this pull request
Dec 20, 2021
* develop: Update error message for redacted exception (streamlit#4200) Release 1.3.0 (streamlit#4192) Wait for enough elements to load in flaky e2e tests (streamlit#4193) Ignore type errors introduced by mypy 0.920 (streamlit#4191) Change st.plotly docstring to fix docs rendering (streamlit#4182) Fix recursion error in repr method of report thread (streamlit#4176)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The core problem is described in
https://docs.cypress.io/guides/core-concepts/retry-ability#Only-the-last-command-is-retried
Chaining
getand then immediatelyeqis prone to flakiness, becauseif the get call sees some but not enough elements, the eq call will
fail, but only the eq call will be retried, so it'll continue to fail
even when the actual page loads the elements before the eq call times
out. Adding an assertion that there are enough elements ensures that the
initial query will be retried until enough elements are present.
This doesn't change all instances of this pattern in the specs, just the ones that have flaked recently. It also doesn't address possible other causes of flaky tests.