-
Notifications
You must be signed in to change notification settings - Fork 713
test: Cleanup Mechanism and ENV Runs for e2e tests #8839
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Explore these optional code suggestions:
|
Testdino Test Results
|
Testdino Test Results
|
Testdino Test Results
|
Testdino Test Results
|
Testdino Test Results
|
Testdino Test Results
|
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 8 | 7 | 0 | 0 | 1 | 88% | 4m 8s |
Testdino Test Results
|
Testdino Test Results
|
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| All tests passed | 21 | 14 | 0 | 0 | 7 | 67% | 4m 41s |
Testdino Test Results
|
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| 94 tests failed | 226 | 95 | 94 | 33 | 4 | 42% | 54m 51s |
Test Failure Analysis
-
pipelines.spec.js: Assertion failures due to unexpected HTTP status codes
- Pipeline testcases should stay on pipeline page if user dismisses the dialog box: Expected status 200 but received 429.
- Pipeline testcases should navigate to dashboard page if user accepts dialog box: Expected status 200 but received 429.
- Pipeline testcases should add source & destination node without connection and error to be displayed: Expected status 200 but received 429.
-
sanity.spec.js: Visibility issues with elements not found
- Sanity Test Cases should display results when SQL+histogram is on and then stream is selected: Element not found - locator('[data-test="logs-search-result-logs-table"]').
- Sanity Test Cases should display histogram in sql mode: Element not found - locator('[data-test="logs-search-result-bar-chart"] canvas').
- Sanity Test Cases should display pagination when only SQL is on clicking and closing the result: Timeout waiting for locator('[data-test="log-table-column-1-_timestamp"]').
-
streaming.spec.js: Timeout issues and query failures
- Streaming for logs No Histogram should be displayed if Data is not available: Element not found - waiting for text 'warning No data found for histogram.'.
- Streaming for logs Enable Streaming for running query after selecting two streams, SQL Mode On and entering Full join queries: Query retrieval failed with an error.
- Streaming for logs Enable Streaming for running query after selecting two streams, SQL Mode On and entering Right join queries: Query retrieval failed with an error.
...and 7 additional failing specs. Visit the TestDino platform for a full breakdown.
Root Cause Analysis
- The changes in .github/workflows/playwright.yml indicate a shift in environment variables and URLs, which may affect test execution.
Recommended Actions
- Investigate the HTTP 429 status in pipelines.spec.js tests to ensure the server can handle requests properly.
- Ensure that the elements referenced in sanity.spec.js are present in the DOM before running visibility checks.
- Review query handling in streaming.spec.js to address the errors related to retrieving search events.
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| 94 tests failed | 226 | 95 | 94 | 33 | 4 | 42% | 54m 44s |
Test Failure Analysis
-
pipelines.spec.js: Multiple timeout issues during user interactions
- Pipeline testcases should stay on pipeline page if user dismisses the dialog box: Received status 429 instead of expected 200.
- Pipeline testcases should navigate to dashboard page if user accepts dialog box: Locator for login user ID not found in time.
- Pipeline testcases should add source & destination node without connection and error to be displayed: Received status 429 instead of expected 200.
-
sanity.spec.js: Visibility issues with elements not found
- Sanity Test Cases should display results when SQL+histogram is on and then stream is selected: Locator for log table column not found.
- Sanity Test Cases should display histogram in sql mode: Locator for histogram canvas not found.
- Sanity Test Cases should display pagination when only SQL is on clicking and closing the result: Locator for timestamp column not found.
-
streaming.spec.js: Errors related to data retrieval and visibility
- Streaming for logs No Histogram should be displayed if Data is not available: Locator for no data warning not found.
- Streaming for logs Enable Streaming for running query after selecting two streams, SQL Mode On and entering Full join queries: Error retrieving search events.
- Streaming for logs Enable Streaming for running query after selecting two streams, SQL Mode On and entering Right join queries: Error retrieving search events.
...and 7 additional failing specs. Visit the TestDino platform for a full breakdown.
Root Cause Analysis
- The changes in .github/workflows/playwright.yml indicate a shift in environment variables and URLs, which may affect test execution.
Recommended Actions
- Investigate the API response handling in pipelines.spec.js to address the 429 status code errors.
- Ensure that all locators in sanity.spec.js are correctly targeting existing elements, particularly for visibility checks.
- Review the data availability conditions in streaming.spec.js to prevent errors when no data is present.
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| 94 tests failed | 226 | 95 | 94 | 33 | 4 | 42% | 55m 0s |
Test Failure Analysis
-
pipelines.spec.js: Multiple tests failing due to timeout and assertion errors
- Pipeline testcases should stay on pipeline page if user dismisses the dialog box: Received status 429 instead of expected 200
- Pipeline testcases should navigate to dashboard page if user accepts dialog box: Locator timeout waiting for login-user-id
- Pipeline testcases should add source & destination node without connection and error to be displayed: Received status 429 instead of expected 200
-
sanity.spec.js: Tests failing due to visibility and timeout issues
- Sanity Test Cases should display results when SQL+histogram is on and then stream is selected: Element not found for logs-search-result-logs-table
- Sanity Test Cases should display histogram in sql mode: Element not found for logs-search-result-bar-chart canvas
- Sanity Test Cases should display pagination when only SQL is on clicking and closing the result: Locator timeout waiting for log-table-column-1-_timestamp
-
streaming.spec.js: Tests failing due to visibility and query errors
- Streaming for logs No Histogram should be displayed if Data is not available: Element not found for 'warning No data found for histogram.'
- Streaming for logs Enable Streaming for running query after selecting two streams, SQL Mode On and entering Full join queries: Query failed while retrieving search events
- Streaming for logs Enable Streaming for running query after selecting two streams, SQL Mode On and entering Right join queries: Query failed while retrieving search events
...and 7 additional failing specs. Visit the TestDino platform for a full breakdown.
Root Cause Analysis
- The changes in .github/workflows/playwright.yml indicate a shift in environment variables and potential configuration issues affecting test execution.
Recommended Actions
- Investigate the API response handling in pipelines.spec.js to address the 429 status errors. 2. Increase the timeout settings in sanity.spec.js for elements that are intermittently not found. 3. Review the query execution logic in streaming.spec.js to ensure proper handling of search events.
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| 11 tests failed | 226 | 161 | 11 | 32 | 22 | 71% | 11m 37s |
Test Failure Analysis
-
pipelines.spec.js: All tests fail due to visibility timeouts on the pipeline name input.
- Pipeline testcases should add source & destination node without connection and error to be displayed: Locator timeout - input[placeholder="Enter Pipeline Name"] not visible.
- Pipeline testcases should display error on entering only source node and save: Locator timeout - input[placeholder="Enter Pipeline Name"] not visible.
- Pipeline testcases should display error on entering only pipeline name and save: Locator timeout - input[placeholder="Enter Pipeline Name"] not visible.
-
pipeline-dynamic.spec.js: Tests fail due to timeouts waiting for the pipeline name input visibility.
- Pipeline Dynamic Stream Names Verify pipeline with dynamic destination name using kubernetes_container_name directly: Locator timeout - waiting for [data-cy="login-user-id"].
- Pipeline Dynamic Stream Names Verify pipeline with dynamic destination name using kubernetes_container_name with underscores: Locator timeout - input[placeholder="Enter Pipeline Name"] not visible.
- Pipeline Dynamic Stream Names Verify pipeline with dynamic destination name using kubernetes_container_name: Locator timeout - input[placeholder="Enter Pipeline Name"] not visible.
-
pipeline-core.spec.js: Tests fail due to visibility timeouts on the pipeline name input.
- Core Pipeline Tests should add source, condition & destination node and then delete the pipeline: Locator timeout - input[placeholder="Enter Pipeline Name"] not visible.
- Core Pipeline Tests should add source & destination node and then delete the pipeline: Locator timeout - input[placeholder="Enter Pipeline Name"] not visible.
...and 2 additional failing specs. Visit the TestDino platform for a full breakdown.
Root Cause Analysis
- The failures are linked to the pipelinesPage.js file where the locator for the pipeline name input is timing out.
Recommended Actions
- Increase the timeout duration in pipelinesPage.js for the pipeline name input visibility check.
- Ensure the input field for the pipeline name is rendered before the tests run in pipelines.spec.js.
- Investigate any recent UI changes that may affect the visibility of the pipeline name input.
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| 9 tests failed | 226 | 164 | 9 | 32 | 21 | 73% | 12m 34s |
Test Failure Analysis
- pipelines.spec.js: Multiple timeout issues related to element visibility and interaction
- Pipeline testcases should add source & destination node without connection and error to be displayed: Locator timeout waiting for '[data-cy="login-user-id"]'.
- Pipeline testcases should display error on entering only source node and save: Locator timeout waiting for 'input[placeholder="Enter Pipeline Name"]' to be visible.
- Pipeline testcases should display error on entering only pipeline name and save: Locator timeout waiting for 'input[placeholder="Enter Pipeline Name"]' to be visible.
- pipeline-dynamic.spec.js: Timeout issues primarily on waiting for element visibility
- Pipeline Dynamic Stream Names Verify pipeline with dynamic destination name using kubernetes_container_name directly: Locator timeout waiting for 'input[placeholder="Enter Pipeline Name"]' to be visible.
- Pipeline Dynamic Stream Names Verify pipeline with dynamic destination name using kubernetes_container_name with underscores: Locator timeout waiting for 'input[placeholder="Enter Pipeline Name"]' to be visible.
- Pipeline Dynamic Stream Names Verify pipeline with dynamic destination name using kubernetes_container_name: Locator timeout waiting for '[data-cy="login-user-id"]'.
- pipeline-core.spec.js: Consistent timeout issues on element interactions
- Core Pipeline Tests should add source, condition & destination node and then delete the pipeline: Locator timeout waiting for '[data-cy="login-user-id"]'.
- Core Pipeline Tests should add source & destination node and then delete the pipeline: Locator timeout waiting for 'input[placeholder="Enter Pipeline Name"]' to be visible.
Root Cause Analysis
- The recent changes in .github/workflows/playwright.yml altered environment variables, potentially affecting element visibility and timeouts.
Recommended Actions
- Increase timeout values in pipelines.spec.js for locators to handle slower responses.
- Verify the visibility and existence of elements before interactions in pipeline-core.spec.js.
- Ensure that the environment variables in .github/workflows/playwright.yml are correctly set to match the expected test environment.
|
| Status | Total | Passed | Failed | Skipped | Flaky | Pass Rate | Duration |
|---|---|---|---|---|---|---|---|
| 9 tests failed | 226 | 166 | 9 | 32 | 19 | 73% | 11m 58s |
Test Failure Analysis
- pipelines.spec.js: All tests fail due to visibility timeouts on the pipeline name input
- Pipeline testcases should add source & destination node without connection and error to be displayed: Locator timeout waiting for input to be visible.
- Pipeline testcases should display error on entering only source node and save: Locator timeout waiting for input to be visible.
- Pipeline testcases should display error on entering only pipeline name and save: Locator timeout waiting for input to be visible.
- pipeline-dynamic.spec.js: Tests fail due to timeouts on the pipeline name input visibility
- Pipeline Dynamic Stream Names Verify pipeline with dynamic destination name using kubernetes_container_name directly: Locator timeout waiting for login user ID input.
- Pipeline Dynamic Stream Names Verify pipeline with dynamic destination name using kubernetes_container_name with underscores: Locator timeout waiting for input to be visible.
- Pipeline Dynamic Stream Names Verify pipeline with dynamic destination name using kubernetes_container_name: Locator timeout waiting for input to be visible.
- pipeline-core.spec.js: Tests fail due to visibility timeouts on the pipeline name input
- Core Pipeline Tests should add source, condition & destination node and then delete the pipeline: Locator timeout waiting for input to be visible.
- Core Pipeline Tests should add source & destination node and then delete the pipeline: Locator timeout waiting for input to be visible.
Root Cause Analysis
- The recent changes in .github/workflows/playwright.yml altered environment variables, potentially affecting input visibility.
Recommended Actions
- Increase the timeout duration in the
enterPipelineNamemethod inpipelinesPage.jsto allow more time for the input to become visible. - Verify the visibility and loading state of the input fields before running tests to ensure they are ready for interaction.
- Check the environment configuration in
.github/workflows/playwright.ymlto ensure it matches the expected test environment.
|
Failed to generate code suggestions for PR |
PR Type
Enhancement, Tests
Description
Add robust alerts cleanup utilities
Improve UI flows and resiliency
Support deployed env testing delays
Add CI cleanup and health checks
Diagram Walkthrough
File Walkthrough
7 files
Add search, cascade-safe delete, bulk prefix cleanupAdd template search, in-use handling, prefix cleanupHarden alert actions, bulk deletion, search across foldersNew API-based cascade cleanup utilityWire APICleanup into PageManagerAdd configurable test step delay hooksHarden login and ingestion with diagnostics1 files
Fix ingestion URLs to use org variable4 files
Pre-move folder cleanup and stabilizationUse dedicated invalid import test dataNew pre-test API cleanup specAdd invalid alert sample for import tests1 files
Target deployed env; add cleanup job and health checks