Summary

Lighthouse is going to slowly introduce support for running on flows, i.e. situations beyond just the initial page load. This issue collects related discussions we've had on this topic in the past, describes the rough roadmap and milestones, and links to relevant design documents as they're written.

Scope

Four core scenarios to eventually support.

• Run snapshot-style audits on a page after interaction, i.e. I've clicked a menu option and now rerun the accessibility category
• Run timespan-style audits on a page during any arbitrary interaction
• Run Lighthouse on a traditional page navigation from an existing page
• Run Lighthouse on a single page app navigation

Tentative Milestones

Phase 0 - Research & Design

• Create an inventory of all audits cataloging their implicit requirements (e.g. snapshot v. timespan v. load) Complete
• Create design doc and project plan Complete

Phase 1 - Basic End-to-End FR Structure

• refactor runner to expose shared components core(runner): abstract gather phase out of runner #11623
• create a driver v2 based on puppeteer page object core(fr): add fraggle rock driver #11742
  • add more limited TransitionalGatherer and TransitionalDriver types that any gatherer supporting FR must use instead of default gatherer (with full access to driver and loadData) core(fr): add fraggle rock driver #11742 core(config): remove gatherer options support #11743
• create a fraggle rock runner that collects afterPass artifacts only core(fr): add base fraggle rock snapshot runner #11748
  • add -pptrtest.js suite support that runs separately from unit but can use real chrome and test puppeteer use cases core(fr): add base fraggle rock snapshot runner #11748

Phase 2 - Configuration Support, Snapshot mode

• create a barebones config v2 that supports artifact definition at the root core(fr): add base config #11915
• refactor all to-be-shared config logic out of config.js into config-helpers.js core(config): refactor config cloning for fraggle rock #11759 core(fr): add base config #11915
• introduce gatherer .meta that can annotate gatherers with their supported mode core(fr): add base gatherer class #11917
• refactor gatherers that support snapshot mode to expose a snapshot method to return the artifact core(fr): add base gatherer class #11917
• add config v2 functionality to filter down to snapshot mode core(fr): filter configs by gather mode #11941

Phase 3 - Timespan mode

• create a basic timespan mode entrypoint in fraggle rock API core(fr): add basic timespan support #11944
  • upgrade the default base gatherer with timespan methods core(fr): add basic timespan support #11944
  • upgrade 1 gatherer (ConsoleMessages might be easiest?) to work with new fraggle rock timespan core(fr): add basic timespan support #11944
  • add the entrypoint method core(fr): add basic timespan support #11944
  • split logic out of api.js into gather/snapshot-runner/gather/timespan-runner core(fr): add basic timespan support #11944

Phase 4 - Navigation mode

• add navigations definition to config
• create minimal navigation mode entrypoint in fraggle rock API
  • skip almost all logic, artifacts and puppeteer APIs only
• create DevToolsLog gatherer
  • extend FR session to handle onAny()
  • expose networkmonitor object on driver / create status monitor on top of recorder
    • cleanup recordFromLogs/getRecords/getRawRecords
• enable non-standard artifact parity
  • add artifact dependency support to config (definition) and runners (sequencing)
  • covert base artifacts to actual artifacts
  • create Trace gatherer and traces shim artifact
  • create devtoolsLogs shim artifact
• reshape navigation mode to match more of gather-runner, FR-COMPAT annotations
  • expose execution context on driver instead of delegate
  • extract driver shared logic @patrickhulce
    • enableRuntime/enableAsyncStacks @patrickhulce
    • register page-side listeners and cached natives @patrickhulce
    • clearDataForOrigin / storage warning @patrickhulce
    • setThrottling @patrickhulce
  • extract gather-runner shared logic @patrickhulce
    • navigation @patrickhulce
    • driver setup (JS dialogs, rIC wrap, etc)
    • phase setup (throttling, scroll position, etc)
    • navigation page load errors
    • slow CPU warning
    • base artifacts
• filter out manual audits in empty categories

Phase 5 - Audit/Artifact upgrading

Goals: Internal refactoring to make places that require a page load more explicit, remove assumptions. (e.g. #11034 #11253 etc)
Goal: Match real-world navigation mode usecases (waitFor conditions, real network monitor stuff)

This part is parallelizable and work likely distributed among the team.

• support TraceOfTab in timespan mode @patrickhulce
  • add gatherMode to audit context @patrickhulce
  • conditionally use main-thread identification depending on mode @patrickhulce
• annotate/upgrade all the gatherers 🎉 see gatherer burndown assignments
• mark audits with violating invariants as notApplicable, see audit burndown assignments

Phase 6 - Transitional Compatibility Layer / Parity / DevTools compat

• Add Fraggle Rock mode to CLI
  • fraggle rock navigation mode
• Test framework for additional cases
• Fraggle Rock Smoketest Parity with Legacy #12861
  • set throttlingMethod devtools by default in timespans
  • protocol timeout support on session
  • preserve devtools log in error cases
  • preserve trace in error cases (ask @paulirish if necessary)
  • OOPIF support
  • config parity
    • navigation extension (via id)
    • plugin support
    • config validation
      • warn and drop audits that don't have artifacts met
• GatherContext as base artifact
• driver.disconnect implementation
• additional trace categories
• timespan emulation
• fix screenshot issue Screenshots with Puppeteer have incorrect dimensions #12988 (not fixable)
• better logging

Phase 7 - Report UI + API Hardening

• API for joining multiple reports
• decide which faked artifacts/LHR properties make the final cut in snapshot/timespan (see ☂️ Breaking changes for v9 #12614 )
• Flow Report UI
  • tagged template literal / JSX / preact helpers
  • summary view
    • category summary grid
  • navigation view
  • timespan report
  • snapshot report
  • help
  • more improvements
    • i18n / pluralize
    • custom report labels (not "Timespan report 1")
• test the bejesus out of it all
  • test real world pages/cross origin navigations that might switch processes of main thread
  • Sample page report(flow): standalone shell report with preact #12850 (comment)
  • i18n string check report(flow): import lhr strings #13215 (comment) tracked elsewhere
• documentation
  • getting started guide / explainer

Phase 8 (Post CDS) - Fraggle Rock as the new normal, scope expansion, cleanup, QoS improvements

High Priority

• DevTools implementation
• navigate from an existing page without URL knowledge i.e. provide navigate: () => myLink.click() instead of a URL
  • navigate without the about:blank jumps
  • remove dependency on requestedUrl and/or backfill requestedUrl to match
• Add Flow Report support to LH viewer + gist (high priority because it will assist with testing)
• Gather/Audit Mode in User Flows #13364

Backlog

Epics

• single page app navigation mode (provide SPA load metrics like navigation, not just timespan) tracked elsewhere
• opportunity rework to focus on kilobytes rather than wastedMs primo prio
• anti-clobber mode (go through our methods that can stomp on puppeteer usage and optionally skip them) tracked elsewhere
  • throttling
  • emulation
  • blocked URL patterns
  • request interception
  • javascript dialogs
  • disconnect should disable all settings
• cleanup tracked elsewhere
  • remove legacy path
    • remove gather-runner
    • remove artifact hacks for
  • update config helpers to use new FR types
• expand FLHR to include interaction steps, labels, etc
• [ ] More consistent thumbnails (example)

Tasks

Roughly priority ranked.

• ensure that defaultSession can't be used to enable/disable domains not doing
• enforce any use of *.disable must use a dedicated session not doing
• Highest-impact audits report(flow): top opportunities #13082 tracked elsewhere
• artifact improvements not doing
  • split cssusage artifact into stylesheet content and usage
  • split jsusage artifact into script content and usage
• mobile viewport for flow report tracked elsewhere
• pagedependencygraph support for timespans fuck no

CDP Asks

• getThrottlingSettings/getEmulationSettings

Related Issues

#1769
#3837
#11312
#11122
#6028
#1418
#2332