Skip to content

Fix flaky Playwright test in test_examples_viz.py#3039

Merged
quaquel merged 2 commits intomesa:mainfrom
vedantvakharia:fix-flaky-viz-test
Dec 29, 2025
Merged

Fix flaky Playwright test in test_examples_viz.py#3039
quaquel merged 2 commits intomesa:mainfrom
vedantvakharia:fix-flaky-viz-test

Conversation

@vedantvakharia
Copy link
Copy Markdown
Contributor

Summary

This PR fixes a persistent CI failure in tests/examples/test_examples_viz.py, specifically affecting the test_pd_grid_model test on the chromium runner.

The failure was caused by a Playwright Strict Mode Violation, where the test runner found multiple images matching the selector when only one was expected.

The Problem

The CI logs reported the following error:

Error: strict mode violation: locator("img") resolved to 2 elements:
    1) <img ... /> aka get_by_role("img").first
    2) <img ... /> aka get_by_role("img").nth(1)

This occurs because the test environment renders multiple visualization components (the Grid view and the Chart view) on the same page. When the test attempts to take a screenshot using page_session.locator("img"), Playwright finds both images and throws an error because the selector is ambiguous.

The Fix

I have updated the run_model_test function to strictly define which image to capture:

  • Grid Capture: Changed .locator("img") to .locator("img").first.

    • Reasoning: The Grid visualization is rendered first at the top of the page.
  • Graph/Chart Capture: Changed .locator("img") to .locator("img").last.

    • Reasoning: The Chart visualization is appended to the page later (when measure_config is present), so it corresponds to the last image element.

This change ensures the selectors are unique, resolving the ambiguity and preventing the test from crashing due to rendering timing or multiple elements

@github-actions
Copy link
Copy Markdown

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 +3.5% [+2.5%, +4.4%] 🔵 +0.5% [+0.4%, +0.6%]
BoltzmannWealth large 🔵 +2.2% [+0.2%, +5.2%] 🔵 +7.9% [+2.7%, +13.3%]
Schelling small 🔵 +0.2% [-0.8%, +1.3%] 🔵 -0.4% [-1.0%, +0.1%]
Schelling large 🔵 +1.3% [+0.1%, +3.1%] 🔵 +1.9% [+0.1%, +4.0%]
WolfSheep small 🔵 +0.6% [-0.3%, +1.2%] 🔵 +1.0% [+0.7%, +1.3%]
WolfSheep large 🔵 -0.4% [-4.1%, +2.0%] 🔵 +3.4% [+2.0%, +4.6%]
BoidFlockers small 🔵 +2.5% [+2.0%, +3.1%] 🔵 +1.5% [+1.3%, +1.8%]
BoidFlockers large 🔵 +2.2% [+1.8%, +2.7%] 🔵 +1.1% [+1.0%, +1.3%]

@quaquel quaquel merged commit e9054ad into mesa:main Dec 29, 2025
14 checks passed
@quaquel quaquel added the ci Release notes label label Dec 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Release notes label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants