Skip to content

test(e2e): wait for app readiness before interactions#65

Merged
steipete merged 2 commits into
mainfrom
codex/stabilize-e2e-tests
Jul 13, 2026
Merged

test(e2e): wait for app readiness before interactions#65
steipete merged 2 commits into
mainfrom
codex/stabilize-e2e-tests

Conversation

@steipete

@steipete steipete commented Jul 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • wait for the mounted app's existing data-connected readiness signal before post-navigation interactions
  • use that readiness boundary across channel-ordering and direct-message reload/navigation paths
  • make Appearance setup retry only its idempotent modal-open/section-select actions while asserting real modal headings
  • reveal the scoped direct-message action through hover and click it normally instead of forcing a hidden control
  • use collision-resistant identities for repeated direct-message test runs

Diagnosis

The named failures are hydration/readiness races, not Go server contention. SSR exposes the sidebar controls before Svelte binds their handlers, so an early settings or mobile-navigation click can be lost; the failing runs still showed fast API responses. The settings modal also refreshes its user state on mount, which can detach the rail once before it settles.

This keeps Playwright's normal timeouts and worker parallelism. It does not add suite retries, forced clicks, sleeps, or worker serialization.

Proof

  • pnpm check
  • pnpm test:e2e — 69 passed
  • Appearance stress: 40 passed, 20 repeats, 16 workers
  • Channel-order stress: 10 passed, 10 repeats, 10 workers
  • Direct-message stress: 20 passed, 20 repeats
  • Full-suite follow-up: 69 passed, 10 workers
  • Autoreview: clean, no actionable findings (full branch and follow-up diff)

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. labels Jul 13, 2026
@clawsweeper

clawsweeper Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 12, 2026, 9:54 PM ET / July 13, 2026, 01:54 UTC.

Summary
The PR adds a shared Playwright app-readiness wait and applies it around navigation, reload, channel-ordering, direct-message, and Appearance settings interactions.

Reproducibility: not applicable. as a user-facing bug reproduction; the PR instead supplies targeted repeated E2E runs showing that the previously flaky interaction paths remain stable after the harness change.

Review metrics: 2 noteworthy metrics.

  • E2E surface: 3 files; 38 added, 3 removed. The patch is narrowly limited to Playwright synchronization helpers and affected call sites.
  • Stress validation: 50 repeated runs passed. The reported Appearance and channel-order stress loops directly exercise the interactions diagnosed as flaky.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Next step before merge

  • No automated repair is needed; the PR has no actionable defect and only requires normal check completion and merge review.

Security
Cleared: The diff only changes Playwright test code and introduces no dependency, workflow, secret, permission, artifact-download, or production execution-path changes.

Review details

Best possible solution:

Land the focused readiness waits after required checks complete, preserving normal Playwright timeouts and parallelism rather than introducing sleeps, forced clicks, suite retries, or worker serialization.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a user-facing bug reproduction; the PR instead supplies targeted repeated E2E runs showing that the previously flaky interaction paths remain stable after the harness change.

Is this the best way to solve the issue?

Yes; waiting on an existing post-mount readiness signal and retrying only idempotent modal setup actions is narrower and more maintainable than sleeps, forced clicks, suite retries, or worker serialization.

AGENTS.md: found, but no applicable review policy affected this item.

Codex review notes: model internal, reasoning high; reviewed against 701505b94733.

Label changes

Label changes:

  • add P3: This is low-risk test-harness stabilization with no production behavior change.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides after-fix live test results for the full E2E suite and high-concurrency repeated runs of the specifically flaky paths.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body provides after-fix live test results for the full E2E suite and high-concurrency repeated runs of the specifically flaky paths.

Label justifications:

  • P3: This is low-risk test-harness stabilization with no production behavior change.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body provides after-fix live test results for the full E2E suite and high-concurrency repeated runs of the specifically flaky paths.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides after-fix live test results for the full E2E suite and high-concurrency repeated runs of the specifically flaky paths.
Evidence reviewed

What I checked:

  • Focused test-only diff: The PR changes three E2E test files, adding 38 lines and removing 3 without modifying production code, dependencies, or workflows. (tests/e2e/app-ready.ts:1, 96fc9f05d8ad)
  • Existing readiness boundary: The new helper waits for the shell's existing data-connected=true state before interacting with controls that SSR may expose before Svelte binds their handlers. (tests/e2e/app-ready.ts:3, 96fc9f05d8ad)
  • Narrow modal retry: Appearance setup retries only modal opening and section selection, scopes assertions to the Account settings dialog, and verifies actual modal headings. (tests/e2e/appearance.spec.ts:8, 96fc9f05d8ad)
  • After-fix stress evidence: The PR body reports pnpm check, 69 passing E2E tests, 40 passing Appearance stress runs, and 10 passing channel-order stress runs. (96fc9f05d8ad)

Likely related people:

  • steipete: Prior merged activity connects steipete to the web UI and its Playwright coverage, including settings-related behavior adjacent to the files changed here. (role: recent area contributor; confidence: medium; commits: 59bc2b6, 96fc9f05d8ad; files: tests/e2e/app-ready.ts, tests/e2e/appearance.spec.ts, tests/e2e/chat.spec.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete merged commit 44463f7 into main Jul 13, 2026
7 checks passed
@shakkernerd
shakkernerd deleted the codex/stabilize-e2e-tests branch July 13, 2026 02:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@steipete