Repository navigation
Upgrade dev dependencies (vitest 5, Playwright 1.63) and fix the collector for Playwright 1.63 - #18
Merged
Merged
Conversation
- vitest 2.1.9 -> 5.0.1 (fixes the critical vitest and high vite/nanoid advisories) - tsx 4.23.1 -> 4.23.15 - @types/node 22.20.1 -> 22.20.4 (dashboard) - package-lock.json regenerated: the old lock pinned vite 5, which vitest 5 cannot peer @playwright/test stays on 1.61.1. Playwright 1.63.0 fails 18 collector tests (INP capture, pagehide flush, route(), Sample counts per Navigation); that needs its own investigation. npm audit: 7 vulnerabilities -> 0. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Content inserted before anything is painted moves nothing, so the browser honestly reports no shift. Under Playwright 1.63 with parallel load, first paint landed 380-728ms after the script ran in about 1 of 12 page loads (measured), so vitals.spec's wait for a layout-shift entry hung to its timeout. The shift now fires 50ms after first-contentful-paint, with a 1s backstop. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…cument) Playwright 1.63 no longer disables Chromium's RenderDocument (PR microsoft/playwright#41303), so every navigation swaps the document's frame. A binding call from the old document's pagehide never reaches Node, and 18 collector tests failed: intermediate Samples, INP and routes were lost. - Test-driven goto/reload/goBack/goForward flush the current document first, with the teardown flush's bounded paint wait, plus a bounded wait for the latest input's event-timing entry (it needs a presented frame). - Link-click and script navigations flush at beforeunload, while the document is still the frame's current one. pagehide stays for engines that keep the frame (Firefox, Chromium without RenderDocument). - A document may report more than once. Node merges its readings by docId into one Sample and writes it at the fixture-driven flush or 400ms after the first reading. - INP is recomputed on every drain, so a later, worse interaction still wins. This also lifts the documented cross-site limitation (renderer-process swap, same mechanism): new crosssite.test.ts pins both Samples and their routes for cross-site goto and link-click journeys. It fails on the old collector. @playwright/test devDependency: 1.61.1 -> 1.63.0. The suite passes on both 1.61.1 and 1.63.0. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
benjithompson
added a commit
that referenced
this pull request
Sep 22, 2026
…ght 1.63 fix (#21) PR #18 fixed the collector for Playwright 1.63 but kept version 0.2.0. npm 0.2.0 was published before that fix. Without a bump, the v0.4.0 release file and npm 0.2.0 would be two different builds with one version number. 0.2.1 gives the fixed build its own version. Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR upgrades the dev dependencies. It also fixes the collector, because Playwright 1.63 broke it. After the upgrade,
npm auditfinds 0 vulnerabilities (before: 7, with 1 critical and 2 high).The
package-lock.jsonfile is regenerated. The old lockfile pinned vite 5, and vitest 5 needs vite 6 or later.Why Playwright 1.63 broke the collector
Cause (measured). Playwright 1.63 no longer turns off Chromium's
RenderDocumentfeature (microsoft/playwright#41303). With RenderDocument on, each navigation gives the new document a new frame. A binding call from the old document'spagehidethen never reaches Node.exposeBindingandaddInitScriptreceives 4 of 4pagehidecalls on 1.61.1 and 0 of 4 on 1.63.0.--disable-features=RenderDocument, 1.63 receives all 4 again.Effect. 18 collector tests failed. Intermediate Samples were lost, INP was missing, and routes landed on the wrong Navigation. Users who run the collector on Playwright 1.63 get the same losses. The peer range
>=1.40.0 <2includes 1.63.Why the collector does not turn RenderDocument off. Playwright does not merge a user's
--disable-featuresflag with its own list, so a second flag can remove Playwright's entries, for examplePaintHolding. Also, RenderDocument is now how Chromium works.The fix (
packages/collector/src/index.ts)pagefixture. It now wrapsgoto,reload,goBackandgoForward, and flushes the current document before each one. The flush uses the same bounded paint wait as teardown. It also waits for the latest input's event-timing entry, for at most 300 ms after that input. That entry exists only after a presented frame, which comes a few ms after the paint (measured).beforeunload. At that time, the old document is still the frame's current document.pagehidestays as a later reading on engines that keep the frame, such as Firefox.docId. It writes the Sample at the fixture-driven flush, or 400 ms after the first reading. The kill test confirms that an intermediate Sample is on disk before a SIGKILL.Additional result: the cross-site limitation is gone. The limitation in the upload kit has the same cause, a frame swap. The new
test/crosssite.test.tschecks both Samples and their routes for a cross-sitegotoand a cross-site link click. It fails on the old collector (4 of 6) and passes with the fix.Test fixture fix (separate commit)
The fixture page inserted its layout-shifting element 50 ms after its script started. Under 1.63 with parallel load, the first paint sometimes came 380 to 728 ms later (measured), so no shift occurred, and the test waited until its timeout. The same problem occurred with the original collector code. The shift now happens 50 ms after first-contentful-paint.
Docs
Test results (measured on macOS, Node 23.11)
npm run typechecknpm auditNotes
EBADENGINEwarning, but the tests pass.goto,reload,goBackandgoForwardnow waits for the flush of the current page first. The wait is usually a few ms. It is bounded by the FCP wait (1.5 s, only when the page never painted), the paint wait (300 ms) and the input wait (300 ms).🤖 Generated with Claude Code