fix: resolve flaky behavior in dashboard tests #8840

ktx-akshay · 2025-10-17T11:25:55Z

PR Type

Bug fix, Tests

Description

Stabilize dashboard chart rendering waits
Replace network idle with canvas detection
Add explicit chart render wait helper
Adjust workflow Node action version

Diagram Walkthrough

flowchart LR
  Tests["Dashboard tests"] -- "add render waits" --> Transpose["Transpose spec"]
  Tests -- "replace networkidle with selector" --> DashboardSpec["Dashboard spec"]
  CI["Playwright workflow"] -- "use setup-node v4" --> NodeSetup["Node action updated"]

File Walkthrough

Relevant files

Tests

dashboard-transpose.spec.js `Add explicit chart render wait in transpose test` tests/ui-testing/playwright-tests/dashboards/dashboard-transpose.spec.js Wait for chart render after apply Update comments to reflect rendering wait	+3/-2
dashboard.spec.js `Use canvas-based wait to reduce flakiness` tests/ui-testing/playwright-tests/dashboards/dashboard.spec.js Replace networkidle waits with canvas selector Add targeted wait for chart canvas attachment Remove redundant networkidle wait	+5/-3

Configuration changes

playwright.yml `Update GitHub Action setup-node to v4` .github/workflows/playwright.yml Downgrade actions/setup-node from v5 to v4 Keep Node.js version at 22	+1/-1

github-actions · 2025-10-17T11:26:50Z

PR Reviewer Guide 🔍

(Review updated until commit `ba8fee1`)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Flakiness Risk After applying transpose, the sequence waits for both a generic chart render helper and a table data load. If the transpose action only affects the table or if chart render does not occur for table-type panels, the additional wait could intermittently time out. Validate that waitForChartToRender consistently resolves for the panel types exercised by this test. // Apply transpose and wait for chart to render await pm.dashboardPanelActions.applyDashboardBtn(); await pm.dashboardPanelActions.waitForChartToRender(); await pm.chartTypeSelector.waitForTableDataLoad(); // Store transposed data after rendering is complete const transposedTableData = await captureTableData(page); Selector Robustness The new wait relies on the presence of a canvas under '[data-test="chart-renderer"]'. If some chart types or responsive states do not use a canvas (e.g., SVG or deferred mount), this selector may fail despite a valid render. Confirm all target charts render a canvas or broaden the condition. // Wait for canvas elements to be rendered await page.waitForSelector('[data-test="chart-renderer"] canvas', { state: "attached", timeout: 15000, });

github-actions · 2025-10-17T11:27:00Z

PR Code Suggestions ✨

Latest suggestions up to ba8fee1
Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Ensure canvas is visibly rendered Waiting for "attached" can pass before the chart is fully painted, causing flakiness. Wait for the canvas to be "visible" and assert a non-zero bounding box after it appears to ensure rendering is complete. tests/ui-testing/playwright-tests/dashboards/dashboard.spec.js [857-860] await page.waitForSelector('[data-test="chart-renderer"] canvas', { - state: "attached", + state: "visible", timeout: 15000, }); +const canvasBBox = await page.locator('[data-test="chart-renderer"] canvas').first().boundingBox(); +expect(canvasBBox && canvasBBox.width > 0 && canvasBBox.height > 0).toBeTruthy(); Suggestion importance[1-10]: 7 __ Why: Switching from "attached" to "visible" reduces flakiness, and adding a non-zero bounding box check strengthens validation. It’s accurate to the new hunk and improves reliability, though not critical.	Medium
General	Wait for actual table change Consecutive waits may overlap and still read stale table data. Replace generic waits with a targeted wait for table rows to update after applying transpose to ensure `captureTableData` reads the transposed state. tests/ui-testing/playwright-tests/dashboards/dashboard-transpose.spec.js [96-99] await pm.dashboardPanelActions.applyDashboardBtn(); await pm.dashboardPanelActions.waitForChartToRender(); -await pm.chartTypeSelector.waitForTableDataLoad(); +// Wait until table row count or a known cell changes to reflect transpose +await page.waitForFunction(() => { + const table = document.querySelector('[data-test="table-renderer"] table'); + if (!table) return false; + const rows = table.querySelectorAll('tr'); + return rows.length > 1 && rows[0].children.length !== rows[1].children.length; +}, { timeout: 15000 }); Suggestion importance[1-10]: 6 __ Why: Targeting a concrete DOM change after transpose can prevent stale reads and improves test robustness. The idea is sound and relevant, but the heuristic may be brittle and app-specific.	Low

Previous suggestions

Suggestions up to commit af931fd

Category	Suggestion	Impact
Possible issue	Ensure chart has non-zero size The readiness check relies on a custom attribute that may be set before the chart is fully sized, causing flaky bounding boxes. Add a post-condition wait to ensure the container has a non-zero size after the instance exists. This avoids intermittent failures when proceeding to `boundingBox()`. tests/ui-testing/playwright-tests/dashboards/dashboard.spec.js [849-855] await page.waitForFunction( () => { - const chartElement = document.querySelector( - '[data-test="chart-renderer"]' - ); - return chartElement && chartElement.hasAttribute("_echarts_instance_"); + const el = document.querySelector('[data-test="chart-renderer"]'); + if (!el \|\| !el.hasAttribute("_echarts_instance_")) return false; + const rect = el.getBoundingClientRect(); + return rect.width > 0 && rect.height > 0; }, { timeout: 15000 } ); -// await page.waitForLoadState("networkidle"); - Suggestion importance[1-10]: 7 __ Why: Strengthening the readiness check to also verify non-zero size can reduce flakiness before calling `boundingBox()`. The snippet aligns with the new hunk and proposes a safe, context-relevant enhancement, though it's an improvement rather than a critical fix.	Medium

testdino-playwright-reporter · 2025-10-17T11:56:04Z

⚠️ Test Run Unstable

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `af931fd`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	364	343	0	19	2	94%	4m 38s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T05:13:18Z

⚠️ Test Run Unstable

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `32151c2`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	17	14	0	2	1	82%	38.4s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T05:58:29Z

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	17	15	0	2	0	88%	30.9s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T06:05:43Z

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	17	15	0	2	0	88%	30.2s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T06:36:53Z

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	17	15	0	2	0	88%	39.5s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T06:41:50Z

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	17	15	0	2	0	88%	39.2s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T07:30:25Z

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	0	0	0	0	0	0%	4.3s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T07:34:20Z

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	0	0	0	0	0	0%	4.3s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T07:45:46Z

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	0	0	0	0	0	0%	4.3s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T08:56:09Z

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	0	0	0	0	0	0%	4.3s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T09:52:25Z

⚠️ Test Run Unstable

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `a23d0c2`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	364	343	0	19	2	94%	4m 39s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T10:05:03Z

⚠️ Test Run Unstable

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `a23d0c2`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	190	178	0	11	1	94%	3m 31s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T10:59:38Z

⚠️ Test Run Unstable

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `ba8fee1`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	365	342	0	19	4	94%	4m 41s

View Detailed Results

testdino-playwright-reporter · 2025-10-21T11:19:55Z

⚠️ Test Run Unstable

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `ba8fee1`

Testdino Test Results

Status	Total	Passed	Failed	Skipped	Flaky	Pass Rate	Duration
All tests passed	365	343	0	19	3	94%	4m 41s

View Detailed Results

github-actions · 2025-10-21T11:33:48Z

Persistent review updated to latest commit ba8fee1

greptile-apps

Greptile Overview

Summary

Replaced unreliable networkidle waits with explicit canvas element waits to fix flaky dashboard tests.

Key Changes:

dashboard.spec.js: removed two waitForLoadState("networkidle") calls, added explicit wait for canvas element attachment before validation
dashboard-transpose.spec.js: added waitForChartToRender() call to ensure chart rendering completes before capturing data
.github/workflows/playwright.yml: downgraded actions/setup-node from v5 to v4 (unrelated to flakiness fix)

Impact:
The changes improve test stability by waiting for specific DOM elements rather than network activity, which is a best practice for Playwright tests. The explicit canvas wait ensures charts are actually rendered before assertions run.

Confidence Score: 4/5

This PR is safe to merge with minimal risk
The changes follow Playwright best practices by replacing networkidle waits with explicit element waits, which should reduce test flakiness. The test logic and validation remain intact. Minor concern about the unrelated workflow downgrade from setup-node v5 to v4.
.github/workflows/playwright.yml - verify the setup-node downgrade is intentional

Important Files Changed

File Analysis

Filename	Score	Overview
tests/ui-testing/playwright-tests/dashboards/dashboard.spec.js	5/5	Replaced networkidle waits with explicit canvas element wait, improving test stability
.github/workflows/playwright.yml	4/5	Downgraded actions/setup-node from v5 to v4 without explanation

Sequence Diagram

sequenceDiagram
    participant Test as Dashboard Test
    participant Page as Browser Page
    participant Chart as Chart Renderer
    
    Note over Test,Chart: Before (Flaky)
    Test->>Page: Click apply dashboard
    Test->>Page: waitForLoadState("networkidle")
    Note over Page: Waits for ALL network to idle
    Note over Page: Can timeout or miss chart renders
    Page-->>Test: Continue
    Test->>Chart: Validate chart rendered
    
    Note over Test,Chart: After (Stable)
    Test->>Page: Click apply dashboard
    Test->>Chart: waitForSelector('[data-test="chart-renderer"] canvas')
    Note over Chart: Explicit wait for canvas element
    Chart-->>Test: Canvas attached
    Test->>Chart: Validate chart rendered

_{3 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2025-10-21T11:35:41Z

.github/workflows/playwright.yml


      - name: Setup Node.js
-        uses: actions/setup-node@v5
+        uses: actions/setup-node@v4


style: downgraded from v5 to v4 - verify this is intentional and not accidentally included in this PR

Prompt To Fix With AI

This is a comment left during a code review. Path: .github/workflows/playwright.yml Line: 330:330 Comment: **style:** downgraded from v5 to v4 - verify this is intentional and not accidentally included in this PR How can I resolve this? If you propose a fix, please make it concise.

…ate comments in transpose test for clarity

…t workflow

github-actions bot added ☢️ Bug Something isn't working 🧹 Updates Review effort 1/5 labels Oct 17, 2025

ktx-akshay force-pushed the e2e-fix-dashbaord-flaky-test branch from 11f89a2 to 848a70f Compare October 21, 2025 08:38

ktx-akshay force-pushed the e2e-fix-dashbaord-flaky-test branch from 3a9861e to a23d0c2 Compare October 21, 2025 09:28

ktx-akshay force-pushed the e2e-fix-dashbaord-flaky-test branch from a23d0c2 to ba8fee1 Compare October 21, 2025 10:36

ktx-akshay requested a review from neha00290 October 21, 2025 11:29

ktx-akshay marked this pull request as ready for review October 21, 2025 11:32

github-actions bot added Review effort 2/5 and removed Review effort 1/5 labels Oct 21, 2025

greptile-apps bot reviewed Oct 21, 2025

View reviewed changes

neha00290 approved these changes Oct 21, 2025

View reviewed changes

refactor: comment out networkidle wait in dashboard tests

1954bf4

ktx-akshay added 4 commits October 27, 2025 10:32

chore: comment out unused test configurations in Playwright workflow

7106cc5

fixed flaky test

e4e3da3

chore: enable specific dashboard tests in Playwright workflow and upd…

ba8c2fd

…ate comments in transpose test for clarity

chore: restore and enable additional test configurations in Playwrigh…

d7db73b

…t workflow

ktx-akshay force-pushed the e2e-fix-dashbaord-flaky-test branch from ba8fee1 to d7db73b Compare October 27, 2025 05:02

ktx-akshay merged commit 1ee9865 into main Oct 27, 2025
71 of 75 checks passed

ktx-akshay deleted the e2e-fix-dashbaord-flaky-test branch October 27, 2025 12:18

fix: resolve flaky behavior in dashboard tests #8840

fix: resolve flaky behavior in dashboard tests #8840

Uh oh!

Conversation

ktx-akshay commented Oct 17, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Type

Description

Diagram Walkthrough

File Walkthrough

Uh oh!

github-actions bot commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Reviewer Guide 🔍

(Review updated until commit ba8fee1)

Uh oh!

github-actions bot commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Code Suggestions ✨

Previous suggestions

Uh oh!

testdino-playwright-reporter bot commented Oct 17, 2025

⚠️ Test Run Unstable

Author: ktx-akshay | Branch: e2e-fix-dashbaord-flaky-test | Commit: af931fd

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

⚠️ Test Run Unstable

Author: ktx-akshay | Branch: e2e-fix-dashbaord-flaky-test | Commit: 32151c2

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

⚠️ Test Run Unstable

Author: ktx-akshay | Branch: e2e-fix-dashbaord-flaky-test | Commit: a23d0c2

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

⚠️ Test Run Unstable

Author: ktx-akshay | Branch: e2e-fix-dashbaord-flaky-test | Commit: a23d0c2

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

⚠️ Test Run Unstable

Author: ktx-akshay | Branch: e2e-fix-dashbaord-flaky-test | Commit: ba8fee1

Testdino Test Results

Uh oh!

testdino-playwright-reporter bot commented Oct 21, 2025

⚠️ Test Run Unstable

Author: ktx-akshay | Branch: e2e-fix-dashbaord-flaky-test | Commit: ba8fee1

Testdino Test Results

Uh oh!

github-actions bot commented Oct 21, 2025

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Greptile Overview

ktx-akshay commented Oct 17, 2025 •

edited by github-actions bot

Loading

github-actions bot commented Oct 17, 2025 •

edited

Loading

(Review updated until commit `ba8fee1`)

github-actions bot commented Oct 17, 2025 •

edited

Loading

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `af931fd`

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `32151c2`

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `a23d0c2`

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `a23d0c2`

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `ba8fee1`

Author: `ktx-akshay` | Branch: `e2e-fix-dashbaord-flaky-test` | Commit: `ba8fee1`