feat(telemetry): move GA emission to Celery beat with refreshed schema by vpetersson · Pull Request #2798 · Screenly/Anthias

vpetersson · 2026-05-01T12:36:57Z

Summary

Move the single GA device_active event off the synchronous request path and into a Celery beat task that ticks hourly.
24h Redis-backed cooldown so each device emits at most one event per day, regardless of how often the worker restarts (Anthias's celerybeat schedule lives in /tmp and resets on every container restart).
Harden the HTTP POST: timeout=5, broad RequestException catch.
Refresh the payload schema for the current device matrix (x86 is now first-class) and add usage signal worth tracking.

Why

The existing GA call lived inline in lib/github.py:is_up_to_date(), which is invoked on the home-page render and the /api/v2/info endpoint. Two separate failure modes:

Dead since the branch list passed 30 — remote_branch_available() paginates on the GitHub API and silently dropped master, so fetch_remote_hash() always returned (None, False) and is_up_to_date() early-returned before reaching the GA block. (Fixed in fix(server): unblock migrations + fix upstream version check #2797.)
Even when it worked, it could deadlock workers — no timeout= on the POST, only ConnectionError was caught, so any HTTPError/Timeout would 500 the request handler. The call was on the sync request path.

Periodic Celery beat is the natural home: off the request path, runs whether or not anyone hits the UI, easy to extend to other periodic metrics later.

Cadence + cooldown

Beat fires hourly (add_periodic_task(3600, send_telemetry_task.s(), name='telemetry'))
send_telemetry() checks telemetry-cooldown in Redis; if set within the last 24h, returns without sending
After a successful send, the cooldown is set with TELEMETRY_COOLDOWN_TTL = 86400
A failed POST does not set the cooldown — the next hourly tick retries

This combination gives at-most-one event per device per 24h. Anthias's celerybeat schedule lives at /tmp/celerybeat-schedule (per docker-compose.yml.tmpl) and resets on every container restart, so a pure interval would never accumulate the 24h gap on devices that reboot daily. The Redis-backed cooldown survives via the persisted redis-data volume.

Schema refresh

Old payload was Pi-centric and missed x86 deployments entirely. New schema (snake_case per GA4 convention):

Field	Source	Why
`branch`, `commit_short`	env / Dockerfile metadata	version identity
`device_type`	`DEVICE_TYPE` env (`pi4-64`/`pi5`/`x86`)	board variant — was missing
`hardware_model`	`/proc/cpuinfo` model	was `Pi_Version`, renamed for x86
`is_balena`, `is_docker`	runtime introspection	deployment flavour
`os_release`	`/etc/os-release` PRETTY_NAME	Debian version drift visibility
`resolution`, `audio_output`	`settings`	viewer config — informs perf/hw decisions like the recent 4K video issue
`tls_enabled`	`settings['use_ssl']`	HTTPS opt-in adoption
`asset_count`, `asset_image_count`, `asset_video_count`, `asset_webpage_count`	DB aggregate	playlist size + content type mix

Dropped:

NOOBS — retired upstream, dead signal
Pi_Version — replaced by neutral hardware_model + device_type pair

Event name changed from version → device_active so the WAU/MAU intent is explicit in GA Explorer.

What you can answer with the new data

WAU / MAU (unique client_id per day, native GA report)
Hardware mix: device_type × hardware_model
Version drift: branch × commit_short
4K vs 1080p adoption: resolution
HTTPS adoption: tls_enabled
Playlist size distribution: asset_count
Content type mix: asset_*_count

Test plan

ruff check + ruff format --check pass
tests/test_telemetry.py covers: cooldown, opt-out, CI gate, timeout/ConnectionError swallowing, device_id reuse + generation, payload shape including new fields
CI green on the PR
On a deployed device: Celery worker logs show the periodic task firing on schedule, GA admin shows live device_active events post-deploy

🤖 Generated with Claude Code

The single GA "version" event used to be emitted inline from is_up_to_date() — which is on the synchronous request path for the home-page banner and the v2 info endpoint — and was gated behind the cache-miss branch of fetch_remote_hash(). Combined with the branches-pagination bug fixed in #2797 it had been dead in practice. Even when it worked, it was prone to deadlocking request workers (no timeout) and crashing them on non-ConnectionError exceptions (narrow except). Move the call to a Celery beat task that fires hourly. The task delegates to lib.telemetry.send_telemetry(), which guards itself with a 24h cooldown stored in Redis (the persisted volume, not the tmpfs celerybeat schedule) so devices that reboot frequently still emit at most one event per day. The HTTP POST gets timeout=5 and a broad RequestException catch so a sluggish GA can never drag down worker latency. Refresh the payload to match Anthias's current device matrix and GA4 conventions: branch, commit_short — version identity device_type — pi4-64 / pi5 / x86 (was missing) hardware_model — /proc/cpuinfo model (was Pi_Version) is_balena, is_docker — runtime flavour os_release — /etc/os-release PRETTY_NAME resolution, audio_output — viewer config that informs hardware/perf decisions tls_enabled — HTTPS adoption signal asset_count, asset_<type>_count — playlist usage breakdown Drops the legacy NOOBS bool — NOOBS was retired upstream years ago and the field has been useless. Drops Pi_Version in favour of the neutral hardware_model + device_type pair so x86 readouts make sense. Lib import note: lib/telemetry pulls in anthias_app.models.Asset for the count query, matching the existing celery_tasks.py pattern (this is a server-side helper, never imported by the viewer). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

device_id is a non-cryptographic identifier — it just needs to be unique enough for GA to count distinct devices — but Sonar flags random.choice with a generic "weak crypto" warning that fails the quality gate. secrets.choice is a one-line drop-in with the same output shape, costing nothing at this volume. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

/etc/os-release inside the anthias-server container is the image's base layer (Debian trixie pinned in the Dockerfile), not the host's OS. The field would be near-constant across every deployment and tells us nothing about user environments, so it's just schema noise. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Anthias only ships as Docker containers; the field would be a constant True across every deployment. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

sonarqubecloud · 2026-05-01T12:45:17Z

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Resolve conflict in tests/test_updates.py — the migration's pytest function-style version supersedes master's stale unittest_parametrize remnant. Convert tests/test_telemetry.py (added on master via #2798) from unittest.TestCase to pytest function-style + fixtures to match the rest of the migrated suite. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

The GA4 "Version distribution" report reads a `version_name` dimension that nothing has ever sent — not this payload, and not the pre-Screenly#2798 one either, whose `Pi_Version` was the *hardware* model. So the report showed ~3,700 devices as "(not reported)" and could never have worked. - add `version_name` from get_anthias_release() (pyproject.toml, which ships in the image) rather than an env var, since missing env vars are the known failure mode for telemetry params. Verified inside anthias-anthias-celery-1 on the pi5 testbed: resolves to '2026.7.3' with no extra plumbing - fall back to 'unknown' rather than '', so a device whose version lookup fails gets its own bucket instead of silently rejoining the "(not reported)" pile that hid this - correct the docstring: the event was renamed `version` -> `device_active` in Screenly#2798 and the docstring still said `version`

The GA4 "Version distribution" report reads a `version_name` dimension that nothing has ever sent — not this payload, and not the pre-#2798 one either, whose `Pi_Version` was the *hardware* model. So the report showed ~3,700 devices as "(not reported)" and could never have worked. - add `version_name` from get_anthias_release() (pyproject.toml, which ships in the image) rather than an env var, since missing env vars are the known failure mode for telemetry params. Verified inside anthias-anthias-celery-1 on the pi5 testbed: resolves to '2026.7.3' with no extra plumbing - fall back to 'unknown' rather than '', so a device whose version lookup fails gets its own bucket instead of silently rejoining the "(not reported)" pile that hid this - correct the docstring: the event was renamed `version` -> `device_active` in #2798 and the docstring still said `version` Co-authored-by: vpetersson-bot <[email protected]>

vpetersson requested a review from a team as a code owner May 1, 2026 12:36

vpetersson and others added 4 commits May 1, 2026 12:39

fix(telemetry): annotate _build_payload return type for mypy

b4a1890

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

fix(telemetry): drop is_docker — always true in our deploy model

3643efd

Anthias only ships as Docker containers; the field would be a constant True across every deployment. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

vpetersson merged commit 49d3ee6 into master May 1, 2026
9 checks passed

vpetersson mentioned this pull request May 1, 2026

refactor(tests): migrate Python test suite to pytest #2801

Merged

8 tasks

This was referenced Aug 7, 2026

fix: bound the CEC subprocess reap, quieten three Sentry noise sources, and report the version to GA4 #3269

Closed

fix(telemetry): send the release version so GA4 can report it #3270

Merged

Navigation Menu

Uh oh!

Uh oh!

feat(telemetry): move GA emission to Celery beat with refreshed schema - #2798

feat(telemetry): move GA emission to Celery beat with refreshed schema#2798
vpetersson merged 5 commits into
masterfrom
feat/telemetry-celery

vpetersson commented May 1, 2026

Uh oh!

sonarqubecloud Bot commented May 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

vpetersson commented May 1, 2026

Summary

Why

Cadence + cooldown

Schema refresh

What you can answer with the new data

Test plan

Uh oh!

sonarqubecloud Bot commented May 1, 2026

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant