Skip to content

[Bug]: supervisord CRIT "could not write pidfile /var/run/supervisord.pid" on rootless podman + keep-id + RO rootfs #580

Description

@enihcam

Describe the bug

On the hardened compose setup shipped with PR #577 (rootless podman + userns_mode: keep-id + read_only: true), supervisord logs a non-fatal CRIT could not write pidfile /var/run/supervisord.pid on every container start. The children still spawn and reach the RUNNING state, so the stack works end-to-end, but every restart fills the log with a CRIT that doesn't match the actual outcome — everything is fine.

Root cause: the compose.yaml ships /var/run as a tmpfs with mode=0755 owned by root, while supervisord's pidfile is created by PID 1, which now runs as UID 1000 (deeptutor) thanks to the non-root-user hardening in PR #577. PID 1 (UID 1000) cannot create a file at /var/run/supervisord.pid because the directory is mode 0755 owned by root.

Steps to reproduce

  1. cp .env.example .env (from the repo root)
  2. podman compose -f compose.yaml up -d
  3. Wait for the deeptutor container to reach the healthy state (backend + frontend on :8001 + :3782, plus the proxy.ts route forward)
  4. podman logs deeptutor | grep -i crit
  5. Observe: CRIT could not write pidfile /var/run/supervisord.pid

Tested against ghcr.io/hkuds/deeptutor:latest on rootless podman 5.8.3, Linux, with userns_mode: keep-id + read_only: true. The same compose runs cleanly under rootful docker / rootful podman where the file does get created.

Expected behavior

No CRIT entries in the logs. Supervisord's pidfile should be writable from PID 1 (UID 1000). Children should still reach the RUNNING state — which they do today, despite the warning.

Configuration Used

The compose.yaml shipped at dev after PR #577 was merged. Both deeptutor and pocketbase services have read_only: true + userns_mode: keep-id. The deeptutor service's tmpfs block currently is:

tmpfs:
  - /tmp:size=512m,mode=1777
  - /run:size=32m,mode=0755
  - /var/run:size=8m,mode=0755     # ← this one
  - /var/log:size=64m,mode=0755
  - /root:size=16m,mode=0700
  - /home:size=16m,mode=0755

Note the existing /tmp:size=512m,mode=1777 already follows the same convention the fix below uses.

Proposed fix (one-line change to compose.yaml)

Change the /var/run tmpfs mode from 0755 to 1777 so PID 1 (UID 1000) can create files there:

- /var/run:size=8m,mode=1777

This matches the project's existing pattern for /tmp (sticky + world-writable) and is the standard convention for runtime / pid dirs.

Acceptance criteria

  • podman compose -f compose.yaml up -d followed by podman logs deeptutor shows zero CRIT lines.
  • podman exec deeptutor cat /var/run/supervisord.pid returns a numeric PID (currently missing — the file isn't created).
  • All existing healthchecks still pass: backend at http://localhost:8001/, frontend at http://localhost:3782/, pocketbase at http://localhost:8090/api/health.
  • Same compose continues to work under rootful docker / rootful podman (no regression).

Additional Information

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions