Skip to content

feat: honor HTTP proxies across install and runtime (#3239) - #3242

Merged
vpetersson merged 3 commits into
Screenly:masterfrom
vpetersson-bot:feat/proxy-support-3239
Aug 5, 2026
Merged

feat: honor HTTP proxies across install and runtime (#3239)#3242
vpetersson merged 3 commits into
Screenly:masterfrom
vpetersson-bot:feat/proxy-support-3239

Conversation

@vpetersson-bot

Copy link
Copy Markdown
Contributor

Closes #3239.

Installing and running Anthias on a host whose only internet path is an HTTP proxy previously meant patching every network layer by hand, because none of them share environment or config: the Ansible play, apt, the Docker daemon, the anthias-host-agent unit, and the app containers each have an isolated environment. Each was verified against the code before this change (the reporter's nine points all held up).

Design — one input, one source of truth, four layers

The proxy is captured once (a new install.sh prompt, prefilled from any HTTP_PROXY/HTTPS_PROXY already in the environment) and propagated from a single on-disk file, /etc/anthias/proxy.env, written by the Ansible system role:

  • install-time tooling — play-level environment: so get_url/apt traverse the proxy; install.sh also exports it for its own curl/git/uv and writes an apt proxy conf (its apt runs via bare sudo, so it never sees the shell env).
  • Docker daemon — a docker.service.d/http-proxy.conf systemd drop-in (EnvironmentFile) so ghcr.io pulls work.
  • host systemd — the one rendered unit (anthias-host-agent) gets an EnvironmentFile; its requests calls then honor it via trust_env.
  • containersHTTP_PROXY/HTTPS_PROXY/NO_PROXY rendered into the server/viewer/celery services via envsubst (upgrade_containers.sh sources proxy.env). All app HTTP clients use requests so they honor it for free; yt-dlp and QtWebEngine inherit the container env.

NO_PROXY is computed, not hand-typed. It always includes the Compose service names (anthias-server, redis) plus loopback, so container-to-container traffic is never force-proxied — the failure that blanked the screen in the report (requests matches NO_PROXY on the URL hostname, so the service names are what matter). Everything is idempotent both ways: with no proxy configured the managed files are removed and rendered values are empty (treated as "no proxy").

Also fixed (robustness points from the report)

  • get_url for the Docker apt key gets a stat+when guard (network-free when the key is already present).
  • boot.yml (writes /boot/firmware/config.txt) gets a device_type when: guard so a direct ansible-playbook run can't hit it on non-Pi hardware.
  • install.sh warns before git reset --hard discards local tracked edits.
  • anthias_user is passed explicitly and falls back to ansible_user_id, not just env USER (which resolves to root under sudo, breaking host-agent paths).

Balena images use a different (supervisor/NetworkManager) proxy mechanism and are out of scope here.

Testing — validated end-to-end against a real logging proxy

Stood up a logging forward proxy and introspected exactly what routed through it vs bypassed it. Hosts seen through the proxy:

  CONNECT cli.github.com          (apt)
  CONNECT download.docker.com     (apt — the exact host that failed in the report)
  CONNECT github.com              (git ls-remote — install.sh clone path)
  CONNECT www.youtube.com         (curl HTTPS)
  CONNECT example.com             (requests + container)
  GET http://archive.ubuntu.com   (apt, x9)
  GET http://security.ubuntu.com  (apt, x3)

Internal service names (anthias-server, redis) in the proxy log: 0 — proven bypass by absence.

Per layer:

  • apt through the exact generated apt.conf → all repos incl. download.docker.com routed via proxy.
  • app clients (requests, trust_env)https://example.com via proxy (200); http://anthias-server:8080 and http://redis:6379 bypassed (no proxy log entry).
  • git / curl HTTP+HTTPS → routed via proxy and succeeded.
  • real container with the compose-rendered env → example.com via proxy (200); anthias-server bypassed (resolved directly, no proxy entry).
  • persistence → sourcing an Ansible-written proxy.env (set -a) + envsubst renders the proxy into all three services.

Static checks all green: bash -n, ansible-lint (production profile), ansible-playbook --syntax-check, and the computed NO_PROXY/environment templating evaluated across proxy / no-proxy / https-only scenarios.

The Docker-daemon drop-in is the standard systemd EnvironmentFile mechanism; the apt run already proves download.docker.com reaches through the proxy, which is the same env the daemon reads for pulls. A full behind-proxy install on the x86 hardware testbed is a sensible pre-merge acceptance check and can follow.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HJ3ucEkn62cbgPoisAZ5LQ

Installing and running Anthias behind an HTTP-proxy-only host previously
required manually patching every network layer separately, because none
of them share environment or config: the ansible play, apt, the Docker
daemon, the anthias-host-agent unit, and the app containers each have an
isolated environment.

Capture the proxy once (a new install.sh prompt, prefilled from any
HTTP_PROXY/HTTPS_PROXY already in the environment) and propagate it to
every layer from a single on-disk source of truth (/etc/anthias/proxy.env,
written by the ansible system role):

- install.sh: prompt early (before prerequisites/require_network); export
  proxy env for its own curl/git/uv; write an apt proxy conf (its apt runs
  via bare sudo and would not see the env); pass the proxy and the install
  user to ansible as explicit --extra-vars.
- ansible: play-level environment: block so get_url/apt traverse the proxy;
  proxy.env source of truth; apt proxy conf; Docker daemon systemd drop-in
  (EnvironmentFile) so ghcr.io pulls work; host-agent unit EnvironmentFile.
- containers: HTTP_PROXY/HTTPS_PROXY/NO_PROXY rendered into the
  server/viewer/celery services via envsubst (upgrade_containers.sh sources
  proxy.env). All app HTTP clients use requests (trust_env) so they honor
  it for free; yt-dlp and QtWebEngine inherit the container env.

NO_PROXY is computed, not hand-typed: it always includes the compose
service names (anthias-server, redis) plus loopback, so container-to-
container traffic never gets force-proxied — the failure that blanked the
screen in the report. requests matches NO_PROXY on the URL hostname, so
the service names are what matter.

Everything is idempotent in both directions: with no proxy configured the
managed files are removed and the rendered env values are empty (which
every client treats as "no proxy").

Also fixes the robustness issues the report surfaced:
- get_url for the Docker apt key gets a stat+when guard (network-free when
  the key is already present).
- boot.yml (writes /boot/firmware/config.txt) gets a device_type when-guard
  so a direct ansible-playbook run can't hit it on non-Pi hardware.
- install.sh warns before `git reset --hard` discards local tracked edits.
- anthias_user is passed explicitly and falls back to ansible_user_id, not
  just env USER (which resolves to root under sudo, breaking host-agent
  paths).

Balena images use a different (supervisor/NetworkManager) proxy mechanism
and are out of scope here.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HJ3ucEkn62cbgPoisAZ5LQ
@vpetersson-bot
vpetersson-bot requested a review from a team as a code owner August 4, 2026 18:25
- Annotate the two 0755 dir modes (/etc/anthias, docker.service.d) with
  NOSONAR: world-traversable dirs are expected here, same as the existing
  /etc/apt/keyrings task (S2612 is a false positive for a traversable dir).
- Use [[ ]] instead of [ ] for the new proxy conditionals in install.sh
  and upgrade_containers.sh (S7688).
- Reuse a local for the repeated --extra-vars literal (S1192).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HJ3ucEkn62cbgPoisAZ5LQ
@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (master@4542ed9). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #3242   +/-   ##
=========================================
  Coverage          ?   90.85%           
=========================================
  Files             ?       76           
  Lines             ?     8361           
  Branches          ?      885           
=========================================
  Hits              ?     7596           
  Misses            ?      548           
  Partials          ?      217           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds end-to-end HTTP(S) proxy propagation for non-balena installs so a single proxy configuration can be applied consistently across install-time tooling (Ansible/apt), the Docker daemon, the host-agent systemd unit, and the runtime containers.

Changes:

  • Adds an interactive proxy prompt in install.sh, persists proxy settings via a new /etc/anthias/proxy.env, and injects proxy variables into Ansible, apt, and container rendering.
  • Configures proxy support for dockerd (systemd drop-in) and the anthias-host-agent service (EnvironmentFile).
  • Ensures proxied installs avoid breaking internal container-to-container traffic by computing NO_PROXY to include anthias-server and redis.

Reviewed changes

Copilot reviewed 10 out of 12 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
docker-compose.yml.tmpl Injects HTTP(S) proxy env vars into server/viewer/celery containers.
bin/upgrade_containers.sh Sources /etc/anthias/proxy.env before envsubst so rendered compose gets proxy values.
bin/install.sh Prompts for proxy settings, exports them for installer tooling, writes apt proxy config, and passes proxy/user vars into Ansible.
ansible/site.yml Adds play-level proxy vars + computed NO_PROXY, and sets play-wide environment: for proxied tasks.
ansible/roles/system/templates/proxy.env.j2 Introduces the persisted proxy “source of truth” env file format consumed by systemd and scripts.
ansible/roles/system/tasks/proxy.yml Manages /etc/anthias/proxy.env and apt proxy config idempotently (create/remove).
ansible/roles/system/tasks/main.yml Includes proxy task setup and adds a non-Pi guard for boot partition tasks.
ansible/roles/system/tasks/docker.yml Avoids unnecessary network fetch for Docker apt key and adds dockerd proxy drop-in management.
ansible/roles/system/handlers/main.yml Adds a handler to restart docker after proxy drop-in changes.
ansible/roles/anthias/templates/anthias-host-agent.service Adds EnvironmentFile=-/etc/anthias/proxy.env so host-agent requests honors proxy.
ansible/roles/anthias/tasks/main.yml Restarts Anthias systemd units when unit templates change so new EnvironmentFile is applied.
ansible/roles/anthias/handlers/main.yml Adds handler to restart Anthias systemd units on template change.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ansible/roles/system/tasks/proxy.yml
A proxy URL can embed credentials (http://user:pass@proxy), so a
world-readable /etc/anthias/proxy.env could leak them to other local
users. Make it 0640, group-owned by the Anthias user so
upgrade_containers.sh (the only non-root reader) can still source it; the
Docker daemon and host-agent EnvironmentFiles are read by systemd as root.
Also removes the NOSONAR — 0640 has no world bits to flag.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HJ3ucEkn62cbgPoisAZ5LQ
@sonarqubecloud

sonarqubecloud Bot commented Aug 4, 2026

Copy link
Copy Markdown

@vpetersson
vpetersson merged commit 599c43f into Screenly:master Aug 5, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proxy is not propagated to Ansible, Docker daemon, anthias-host-agent, or containers during installation behind an HTTP proxy

3 participants