test(e2e): add Playwright smoke test suite for core OL pages by mekarpeles · Pull Request #12998 · internetarchive/openlibrary

mekarpeles · 2026-06-22T03:57:25Z

Summary

Adds a Playwright end-to-end smoke test suite covering 7 core Open Library pages. Tests run against a local Docker instance or any deployed OL environment via OL_BASE_URL.

Pages covered

Spec	Tests	Notes
`home.spec.ts`	4	Title, header, search trigger, footer
`login.spec.ts`	6	Form fields, submit, signup link, invalid creds, mobile layout
`search.spec.ts`	5	Results load, result items, stats, empty query, form present
`work.spec.ts`	5	Title, work details, cover, editions link, mobile layout
`subjects.spec.ts`	5	Heading, book count, content body, search form, unknown subject
`author.spec.ts`	6	Name heading, content body, works section, works list, unknown author, mobile
`edition.spec.ts`	7	Title, work details, cover, work link, edition metadata, unknown edition, mobile

All test.describe blocks are tagged @smoke — run targeted with:

OL_BASE_URL=http://localhost:8080 npx playwright test --grep "@smoke"

Test design

Desktop Chrome + Pixel 5 (Chromium) mobile projects
Solr-dependent tests skip gracefully when Solr is unavailable/unhealthy
Author/edition tests skip gracefully when the entity isn't in the local DB
Console error collection via shared helpers.ts (filters known env noise)

Running locally

cd ~/Projects/openlibrary  # or any worktree
OL_MOUNT_DIR="$(pwd)" docker compose up -d web infobase db memcached home covers
cd ~/Projects/openlibrary-12885-playwright-e2e
npx playwright install chromium
OL_BASE_URL=http://localhost:8080 npx playwright test --grep "@smoke"

Verified

22 passed, 16 skipped (mobile + Solr-dependent), 0 failed against local Docker

Closes #12885

accesslint

Found 1 issue across 1 rule.

accesslint · 2026-06-22T03:57:33Z

+    test('header search trigger is present', async ({ page }) => {
+        await page.goto('/');
+        // The search bar is a Lit web component — the visible affordance is a trigger button
+        // that opens a search dialog; there is no plain <input> in the DOM until the dialog opens.


WCAG 4.1.2: Form element has no accessible label.

Form elements must have labels. Use <label>, aria-label, or aria-labelledby.

Details

Every form input needs an accessible label so users understand what information to enter. Use a <label> element with a for attribute matching the input's id, wrap the input in a <label>, or use aria-label/aria-labelledby for custom components. Placeholders are not sufficient as labels since they disappear when typing.

- home: search bar is a Lit component; replace input[type=text] locator with .search-bar-component (the trigger button is what's in the DOM) - subjects: skipIfNoSolr only guarded on 404; also skip if #header-bar is absent (Solr unhealthy returns 200 error page); fix .contentBody → #contentBody; add skip guard to the unknown-subject 500 test (Solr down breaks that path too)

Add @axe-core/[email protected] and create tests/e2e/a11y/ with: - axe-helpers.ts: shared AxeBuilder factory (WCAG 2.1 AA tags) and violation formatter for test annotations - a11y-scan.spec.ts: 6 core pages tagged @a11y; annotates violations but does not fail — baseline is 218 errors (2026-06-23); hardening comment explains the per-class upgrade path as fixes land GH: #13007

Playwright axe-core test covering the fix in PR #13011: ReadButton.html <summary> elements must have an accessible name. Tests search results, author works, and work pages — all pages that render the ReadButton macro with listen/locate options. Currently fails against openlibrary.org (fix pending #13011 merge); will pass once the fix deploys.

Playwright axe-core tests covering the fix in PR #13012: - login page: ia_thirdparty_logins iframe must have title - work page: BookPreview iframe must have title - signup page: OL iframes must have title (reCAPTCHA excluded) All three tests currently fail against openlibrary.org (expected — fix pending #13012 merge); will pass once the fix deploys.

openlibrary-bot · 2026-06-24T07:31:55Z

Hi — I'm Pierre, an AI-assisted PR tidier running on behalf of @mekarpeles.

I rebased this branch onto the latest master (was 306 commits behind).

What changed: Only the rebase itself — no content changes. The one conflict was in package.json (scripts section) and package-lock.json (a node_modules/pkg-dir entry). Resolved by keeping the branch's new scripts (test:e2e, storybook, build-storybook) and master's lockfile for the conflicting block (since master's devDependencies already includes @axe-core/playwright).

Why: Brings the branch up to date so GitHub can compute mergeability and CI can run against current master.

Safeguards: All 12 commits on this branch are authored by @mekarpeles. Zero unresolved review threads before proceeding. Used git push --force-with-lease.

To reverse: Run git reflog locally to find the SHA before this push (old tip was 809604343), then git reset --hard 809604343 and git push --force-with-lease to restore.

Playwright axe-core tests covering the fix in PR #13027: - search page: OlToggle sublabel must pass 4.5:1 contrast - work page: nav-bar selected tab must pass 4.5:1 contrast Both tests currently fail against openlibrary.org (expected — fix pending #13027 merge); will pass once the fix deploys.

Regression guard for PR #13029: the generic-dropper arrow anchor (my-books reading-list dropdown trigger on search + author pages) must have an accessible label per WCAG 2.4.4. Tests will fail until #13029 merges and deploys.

Regression guard for PR #13031: home page carousel slides must not have role=option (nested-interactive WCAG 4.1.2). Test currently passes against local env with fix applied.

…opover

lokesh · 2026-07-01T06:38:25Z

Heads up @mekarpeles — I've opened #13072 as a scoped-down slice of this branch to get Playwright landed with minimal risk, and I'd suggest we move forward with that one and repurpose this PR for the a11y follow-up.

What #13072 keeps from here: the core-page smoke specs, the collectConsoleErrors helper + noise filter, and the two-project Playwright config — all descended from this branch.

What it changes:

Trimmed to the "page loads + no console errors" anchors (~20 checks, 11 files) so it's easy to review and land.
Fixes the desktop/mobile project split — this branch's config has no grep filter, so every test runs under both projects; test(e2e): add minimal Playwright smoke suite for core OL pages #13072 uses grep/grepInvert @mobile so nothing runs twice.
Drops the unrelated storybook/build-storybook scripts so the PR is Playwright-only.
It's rebased clean on master and green; this branch is currently conflicting and 300+ commits behind.

What stays here / comes back as its own PR: the axe-core a11y suite (tests/e2e/a11y/*) and the @axe-core/playwright dep. Those guards reference fixes still in flight (#13008, #13009, #13037), and the top-level scan is "infrastructure only" (doesn't fail on violations yet) — so they're more valuable landing after their fixes, rebased on top of #13072, where each rule can actually assert zero and mean it.

Proposed sequence: merge #13072 → rebase the a11y work here (or a fresh PR) on top of it → gate each rule as its fix confirms on master. Happy to do that rebase. Sound good?

Introduces Playwright as a dev dependency and a small, manually-run smoke suite covering 7 core pages (home, login, search, work, subjects, author, edition). Tests run against local Docker or any OL_BASE_URL via `npm run test:e2e`. Intentionally minimal first step for a new dev tool: - NOT wired into CI and does not block anything yet - no axe-core / a11y scans (deferred to a follow-up) - no Storybook scripts (out of scope) Scoped down from #12998. Refs #12885

* fix(a11y): add title attributes to iframes (WCAG 2.4.1 H64) Iframes must have a title attribute so screen readers can identify their purpose before deciding whether to enter them. - ia_thirdparty_logins.html: title="Sign in with a third-party account" (IA social login widget rendered on login and signup pages) - BookPreview.html: title="Book preview" (floating preview dialog, JS-populated on demand) Excluded: reCAPTCHA iframe on signup page is third-party JS (Google); its title is not under OL template control. Playwright regression test staged in #12998 (tests/e2e/a11y/frame-title.spec.ts). Fixes: frame-title axe rule / WCAG 2.4.1 H64 Closes: #13008 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

…enu (WCAG 1.4.3) (#13027) * fix(a11y): fix color contrast violations in OlToggle and nav-bar (WCAG 1.4.3) Two WCAG 1.4.3 (Contrast Minimum) violations fixed: OlToggle.js: sublabel used hardcoded #777 (contrast 4.47:1, fails AA). Changed to var(--accessible-grey) which is already defined in tokens as the minimum accessible grey (hsl 0, 0%, 46.3% = 4.54:1 against white). nav-bar.css (work-menu): selected tab used --blue-5e88B9 background (hsl 212, 39%, 55%) with white text — contrast only 3.4:1. Changed to --primary-blue (hsl 202, 96%, 37%, 4.82:1) for selected state and --header-nav-hover-color (hsl 207, 83%, 33%, 6.56:1) for hover. Note: .star.star--small gold color contrast (pa11y-only, not axe) is a design decision tracked separately in #13009. Playwright regression tests staged in #12998 (tests/e2e/a11y/color-contrast.spec.ts). Fixes: color-contrast axe rule / WCAG 1.4.3 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

lokesh · 2026-09-02T16:35:51Z

Closing this one out — the work here has since landed through a few smaller PRs.

Where it went:

test(e2e): add minimal Playwright smoke suite for core OL pages #13072 — the Playwright smoke suite (the scoped-down slice of this branch)
feat(a11y): wire @axe-core/playwright into the e2e suite #13251 — wired @axe-core/playwright in, so the e2e suite runs real accessibility scans against rendered pages
e2e: cover logged-in state; fix rating-button contrast and login-page JS error #13366 — added logged-in coverage, and caught + fixed a real colour-contrast bug on the rating buttons

So playwright.config.ts, tests/e2e/helpers.ts, and the page smoke specs are all on master already.

The one bit that didn't come along is the set of per-rule regression guards — aria-prohibited-attr, frame-title, link-name, nested-interactive, summary-name. Those are still worth having, and the pattern for writing them is documented in tests/e2e/a11y.ts on master, so they'd make an easy follow-up PR.

@mekarpeles if you'd rather keep this open, please just reopen — no objections from me.

accesslint Bot reviewed Jun 22, 2026

View reviewed changes

This was referenced Jun 23, 2026

fix(a11y): add aria-label to ReadButton dropdown summary (WCAG 4.1.2) #13011

Closed

fix(a11y): add title attributes to iframes (WCAG 2.4.1 H64) #13012

Merged

mekarpeles added 12 commits June 24, 2026 01:29

test(e2e): add Playwright config and console-error helper

b5b843e

test(e2e): add home page smoke tests

e25c5ab

test(e2e): add login page smoke tests

e46f99d

test(e2e): add search page smoke tests

2cce392

test(e2e): add work (book) page smoke tests

7c31431

test(e2e): add subjects page smoke tests

ad37ebb

test(e2e): add author page smoke tests

8644468

test(e2e): add edition page smoke tests

3fbdfe8

mekarpeles force-pushed the 12885/playwright-e2e-tests branch from 8096043 to 7c9dd64 Compare June 24, 2026 07:31

This was referenced Jun 24, 2026

fix(a11y): fix color contrast in OlToggle sublabel and nav-bar work-menu (WCAG 1.4.3) #13027

Merged

fix(a11y): mark decorative star rating symbols aria-hidden (WCAG 1.4.3) #13028

Merged

mekarpeles mentioned this pull request Jun 24, 2026

fix(a11y): add aria-label to generic-dropper arrow button (link-name WCAG 2.4.4) #13029

Merged

3 tasks

mekarpeles mentioned this pull request Jun 24, 2026

fix(a11y): remove role=option from slick carousel slides (nested-interactive WCAG 4.1.2) #13031

Closed

4 tasks

test(a11y): add nested-interactive regression guard for slick carousel

acc9149

Regression guard for PR #13031: home page carousel slides must not have role=option (nested-interactive WCAG 4.1.2). Test currently passes against local env with fix applied.

github-actions Bot added the Needs: Response Issues which require feedback from lead label Jun 24, 2026

test(a11y): add aria-prohibited-attr regression guard for ol-select-p…

ecea10d

…opover

lokesh mentioned this pull request Jun 30, 2026

test(e2e): add minimal Playwright smoke suite for core OL pages #13072

Merged

lokesh mentioned this pull request Jul 3, 2026

fix(a11y): add aria-label to ReadButton dropdown summary (WCAG 4.1.2) #13116

Merged

lokesh closed this Sep 2, 2026

Navigation Menu

Uh oh!

Uh oh!

test(e2e): add Playwright smoke test suite for core OL pages - #12998

test(e2e): add Playwright smoke test suite for core OL pages#12998
mekarpeles wants to merge 16 commits into
masterfrom
12885/playwright-e2e-tests

mekarpeles commented Jun 22, 2026

Uh oh!

accesslint Bot left a comment

Uh oh!

accesslint Bot Jun 22, 2026

Uh oh!

openlibrary-bot commented Jun 24, 2026

Uh oh!

lokesh commented Jul 1, 2026

Uh oh!

lokesh commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

mekarpeles commented Jun 22, 2026

Summary

Pages covered

Test design

Running locally

Verified

Uh oh!

accesslint Bot left a comment

Choose a reason for hiding this comment

Uh oh!

accesslint Bot Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

openlibrary-bot commented Jun 24, 2026

Uh oh!

lokesh commented Jul 1, 2026

Uh oh!

lokesh commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants