feat(webview): per-asset custom HTTP headers for web assets - #3162
Conversation
Adds a per-asset setting that injects custom request headers (e.g. an
Authorization token for a private Grafana dashboard) into QtWebEngine
when a webpage asset is displayed. Works on both Qt5 and Qt6 via one
non-gated code path. Closes the request in the linked feature issue.
- Store headers in Asset.metadata['headers'] as {name: value}; no
migration (mirrors the refresh_interval_s feature end-to-end)
- Validate/sanitise header names (RFC 7230 token) and reject CR/LF/NUL
in values to prevent header splitting; cap count and length
- v2 API: custom_headers read + write fields folded into metadata
- Edit modal: webpage-only "Custom HTTP headers" textarea, one
Name: Value per line, parsed + clamped in assets_update
- Viewer sends headers as JSON over a new setRequestHeaders D-Bus slot
(version-skew latched like setReloadInterval); reloads on header change
- C++ RequestHeaderInterceptor on the shared QWebEngineProfile, scoped
to the asset's own host so tokens never leak to third-party requests
- Fix host-only test isolation: move shared redis/dbus/env mocks to a
root conftest.py so the api/tests tree inherits them, and reset the
ViewerPublisher/ReplyCollector singletons per test
- Add unit, v2 API, and viewer D-Bus tests
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
There was a problem hiding this comment.
Pull request overview
Adds end-to-end support for per-asset custom HTTP request headers on webpage assets, persisted in Asset.metadata['headers'] and injected at request time by the QtWebEngine webview (Qt5 + Qt6) with API/UI/viewer wiring and tests. Also refactors pytest configuration so API tests run in isolation with the same Redis/D-Bus/environment stubbing as the main test tree.
Changes:
- Add model helpers + validations for header parsing/normalization and expose the feature via v2 API create/update fields.
- Extend the viewer ↔ webview D-Bus interface with
setRequestHeadersand implement a QtWebEngine request interceptor to attach headers for webpage assets. - Move shared pytest isolation to repo-root
conftest.pyand add/extend unit + API + viewer tests for the feature.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_viewer.py |
Adds viewer-side tests for sending/clearing headers, reload-on-header-change, and version-skew latch behavior. |
tests/test_asset_headers.py |
New unit tests for header parsing/normalization/validation helpers. |
tests/conftest.py |
Keeps only Playwright/marketing fixtures; shared isolation moved to repo root. |
conftest.py |
New repo-root pytest isolation: ENV var, gi/pydbus stubs, fake Redis, singleton seeding/reset. |
src/anthias_server/app/models.py |
Adds header validation/normalization/parsing helpers and limits. |
src/anthias_server/api/serializers/v2.py |
Adds custom_headers read field + strict write validation and metadata merge behavior. |
src/anthias_server/api/tests/test_assets.py |
Adds v2 API round-trip + rejection + sanitization tests for custom headers. |
src/anthias_server/app/views.py |
Parses custom_headers textarea for the HTML form and stores into metadata['headers']. |
src/anthias_server/app/templatetags/asset_filters.py |
Sanitizes metadata.headers before feeding the edit modal. |
src/anthias_server/app/templates/_asset_modal.html |
Adds webpage-only “Custom HTTP headers” textarea to the edit modal. |
src/anthias_server/app/static/src/home.ts |
Extends asset edit typing to include optional metadata.headers. |
src/anthias_viewer/__init__.py |
Sends headers over new D-Bus slot, caches applied headers, and reloads when headers change. |
src/anthias_webview/src/view.h |
Declares setRequestHeaders and stores staged headers + interceptor pointer. |
src/anthias_webview/src/view.cpp |
Implements request interceptor + JSON parsing + per-load scoping before navigation. |
src/anthias_webview/src/mainwindow.h |
Exposes setRequestHeaders as a D-Bus slot on MainWindow. |
src/anthias_webview/src/mainwindow.cpp |
Forwards setRequestHeaders to View. |
src/anthias_webview/src/main.cpp |
Updates D-Bus export comment to include setRequestHeaders. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Hardware testing on x86/Qt6 surfaced a race: the first setRequestHeaders D-Bus call after a cold webview spawn can lose to the webview's D-Bus registration and fail transiently. On a single-asset playlist the unchanged-URL short-circuit then skips loadPage forever, so the C++ interceptor never receives the staged headers and the page loads without them. - _apply_request_headers now returns whether the headers are in effect - view_webpage only caches current_browser_headers on success, so a transient failure leaves the mismatch in place and the next asset_loop tick re-issues both setRequestHeaders and loadPage - add a regression test for the transient-retry path Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The root conftest that carries the Redis/D-Bus/env isolation is also an ancestor of tools/raspberry_pi_imager/tests, which the run-python-linter job runs with -p no:django in a minimal venv (no Django, no pytz). Its import-time connect_to_redis patch pulled in anthias_common.utils -> pytz and crashed conftest collection for those app-free tests. Gate every app-dependent hook (connect_to_redis patch, _mock_redis and _ensure_assetdir autouse fixtures) on an import probe so the file degrades to a no-op when the app isn't importable, while still providing full isolation for the app test suites. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Hardware validation (Qt5 + Qt6)Built the viewer image from this branch for both a Qt6 and a Qt5 target and exercised the real compiled interceptor on testbeds (header injection observed at an echo server that logs request headers). Qt6 — full E2E ✅ (x86 testbed)Built Path exercised end-to-end: Django form/API → This surfaced (and I fixed, commit Qt5 — compile + interceptor runtime ✅, final render blocked by headless HW
|
Addresses PR review (Copilot + CodeQL): - Scope custom headers to the asset's full origin (scheme + host + port) instead of host only, so an Authorization header can't ride an https->http downgrade or reach a different port on the same host. - Parent the request interceptor to the process-lifetime default profile (not the View) and detach it in ~View, removing a potential use-after-free where the profile could outlive the interceptor. - v2 custom_headers validation raises a fixed, self-authored message rather than echoing the caught exception text (CodeQL "information exposure through an exception"). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Follow-up: Qt5 render attempt on a real Pi 3, and the origin-scoping re-checkAfter the review fixes (commit Qt6 (x86) — still full E2E ✅ with the origin-scoped interceptor: a same-origin webpage asset's request carried Qt5 render on a real Pi 3 — blocked by the test fleet's GPU stack Root cause: the armhf Qt5 WebView is built for the legacy 32-bit Raspbian brcm/dispmanx GL stack, but every board in the test fleet runs a 64-bit KMS OS, where that GL path can't bind — so QtWebEngine gets no GL context. Reproduced identically on Pi 4 and the Pi 3. This is an environment mismatch, not a code issue. Net Qt5 evidence: compiles cleanly under Qt 5.15 ( All three testbeds were restored to their prior images and locks released. |
…s dict Copilot re-review: assigning current_browser_headers = headers aliased the caller's dict, so a later in-place mutation could make the next value comparison spuriously equal and skip a needed reload. Store dict(headers) instead. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Copilot re-review (security): on a transient setRequestHeaders failure during an asset transition, view_webpage still issued loadPage. The C++ side then reused the PREVIOUS asset's staged headers and scoped them to the new asset's origin — leaking e.g. a prior asset's Authorization header to the next asset's host. Now loadPage is skipped entirely when the header update didn't apply; the URL/header cache is left stale so the next asset_loop tick retries once the D-Bus call succeeds. Supersedes the earlier "cache only on success" tweak. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Copilot re-review: DictField(child=CharField(...)) coerced numeric /
boolean JSON values to strings ({"X": 123} -> "123") before
validate_asset_headers could reject them, so the strict validation was
bypassed. Drop the CharField child (use the unvalidated child) on both
the create and update serializers so values reach validate_asset_headers
verbatim and its isinstance(str) gate 400s non-strings. Also avoids
CharField trimming a value we forward on the wire byte-for-byte.
Adds numeric / bool / null rejection cases to the API test.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Copilot re-review (8 doc nits): the origin-scoping change left comments, docstrings, and the edit-modal copy still describing "same-host" / "same-site" scoping. Align all of them with the implementation (same-origin = scheme + host + port), and correct the root conftest comment to note ENVIRONMENT/dbus-stub setup still runs while only the app-dependent hooks no-op when the app stack is absent. No logic change. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…undary Copilot re-review: - conftest app-availability probe caught bare Exception, which would silently disable Redis isolation for the whole app suite on any real import error. Narrow to ImportError (the missing-pytz/Django case); let genuine app import failures surface. - C++ setRequestHeaders forwarded arbitrary JSON names/values into info.setHttpHeader() unchecked. Add defensive re-validation at this trust-no-one D-Bus boundary (RFC 7230 name token, reject CR/LF/NUL values, cap count/lengths) mirroring validate_asset_headers, so a hostile/buggy caller can't put splitting bytes on the wire even though the server side is the primary gate. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Copilot re-review: - MAX_HEADER_VALUE_LEN now caps the UTF-8 byte length (value.encode), not the Python character count, matching the webview's byte-based cap (values go on the wire as UTF-8). - models.py comment updated to same-origin, and to note the server side is the primary gate with the webview re-validating defensively. - conftest docstring: correct the module path to src/anthias_viewer/__init__.py. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|



What & why
Resolves the feature request for saving and sending custom HTTP headers on web (webpage) assets — for example an
Authorizationheader carrying a Grafana service-account token, so a private dashboard renders on screen without having to be made public.Adds a per-asset setting whose headers are injected into QtWebEngine when the asset is displayed. It works on both Qt5 (Pi 1–3) and Qt6 (Pi 5 / x86) through a single, non-version-gated code path.
How it works
Headers are stored in
Asset.metadata['headers']as a{name: value}object (no schema migration — mirrors the existingrefresh_interval_sfeature end to end). AQWebEngineUrlRequestInterceptorinstalled on the shared profile attaches them at request time, scoped to the asset's own host so a bearer token is never sent to a third-party CDN/analytics/font domain the page also talks to — while same-host XHR (which a dashboard needs to render its panels) still carries it.The chain:
app/models.py) —normalize_asset_headers/validate_asset_headers/parse_header_lines. Header names must be RFC 7230 tokens; CR/LF/NUL are rejected in values to prevent header/response splitting; count and length are capped.api/serializers/v2.py) —custom_headersas a read field (SerializerMethodField) and a strict write field on create/update, folded intometadata['headers'](400 on malformed input)._asset_modal.html,views.py,asset_filters.py,home.ts) — a webpage-only "Custom HTTP headers" textarea, oneName: Valueper line, parsed and clamped server-side (the form forgives; the API 400s).anthias_viewer/__init__.py) —view_webpage(headers=...)sends the headers as JSON over a newsetRequestHeadersD-Bus slot, version-skew-latched exactly likesetReloadInterval; a header-only edit still forces a reload.anthias_webview/src/*) —RequestHeaderInterceptor(mutex-guarded, since Chromium may call it off the UI thread on Qt5), wired throughView/MainWindow.Testing
ruff check+ruff format --checkclean.Incidental fix: host-only test isolation
Running
pytest src/anthias_server/api/tests/in isolation produced 12 spurious 500s. The shared redis/dbus/env test isolation lived only intests/conftest.py, which pytest never loads for the API test tree (no common ancestor), andViewerPublisher/ReplyCollectorare unreset singletons that leak a real redis client across tests. CI masked this because theredisservice resolves there. Fixed by moving the shared isolation into a rootconftest.py(both trees inherit it) and seeding those singletons with the per-test fake. Isolated API run: 12-failed/56s → 96-passed/5.8s; full suite unchanged.Follow-up
🤖 Generated with Claude Code