Skip to content

feat(deploy): harden image for rootless podman with read-only rootfs#577

Merged
pancacake merged 8 commits into
HKUDS:devfrom
enihcam:feature/rootless-podman-hardening
Jun 20, 2026
Merged

feat(deploy): harden image for rootless podman with read-only rootfs#577
pancacake merged 8 commits into
HKUDS:devfrom
enihcam:feature/rootless-podman-hardening

Conversation

@enihcam

@enihcam enihcam commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Closes #576.

This PR hardens the published ghcr.io/hkuds/deeptutor image for rootless podman with read-only rootfs, and ships a tested compose.yaml for that deployment shape. The change set is intentionally small: the architectural core (URL knowledge moves from build-time sed to request-time proxy rewriter) plus the minimum plumbing to make it work.

What's actually changing

The image:

  • Bakes a non-root deeptutor user (UID 1000); the entrypoint drops privileges via gosu after chowning /app/data.
  • start-frontend.sh no longer mutates the bundle with sed -i (which broke under RO rootfs). URL knowledge moves to web/proxy.ts, which rewrites /api/* and /ws/* to DEEPTUTOR_API_BASE_URL at request time.
  • entrypoint.sh exports DEEPTUTOR_API_BASE_URL and DEEPTUTOR_AUTH_ENABLED from data/user/settings/*.json on every start.

The frontend:

  • web/lib/api.ts: apiUrl and wsUrl become (p) => p pass-throughs (the browser fetches relative paths through the frontend; the proxy does the cross-origin work). All existing call sites stay as-is.
  • web/proxy.ts: added the rewrite block + reads DEEPTUTOR_AUTH_ENABLED for the auth-redirect check.

Deployment:

  • docker-compose.yml: pocketbase mount path corrected (/pb_data, /pb_public, /pb_hooks — no /pb/ prefix, matching the upstream image's entrypoint).
  • compose.yaml (new): tested rootless-podman config with read_only: true, userns_mode: keep-id, tmpfs, and bind mounts.
  • .env.example (new): companion env file for compose.yaml.

Docs:

  • CONTAINERIZATION.md (new): full per-installation guide.
  • README.md: one-line pointer.

Commits

<hash>  feat(docker): harden image for rootless/RO rootfs + drop runtime sed
<hash>  fix(docker-compose): correct pocketbase mount path
<hash>  feat(deploy): add podman compose.yaml + .env.example for rootless
<hash>  docs(containerization): add CONTAINERIZATION.md + README pointer

Verification

  • npm run lint, tsc --noEmit, npm run test:node all clean (119 node tests pass; the deleted api-resolve-base.test.ts is replaced by 3 apiFetch redirect tests on the new DEEPTUTOR_AUTH_ENABLED env var).
  • pre-commit run --all-files clean.
  • podman build -t deeptutor:pr-test -f Dockerfile --target production . succeeds.
  • The architectural change is verified: the bundle no longer contains the __NEXT_PUBLIC_API_BASE_PLACEHOLDER__ token (the .env.local block is gone), and apiUrl/wsUrl resolve to identity functions. proxy.ts reads DEEPTUTOR_API_BASE_URL (set by the entrypoint) and rewrites /api/* and /ws/* to it.

Known minor follow-up

supervisord emits CRIT could not write pidfile /var/run/supervisord.pid on startup because /var/run tmpfs is mode=0755 (owned by root) and the now-unprivileged PID 1 can't write there. Children still spawn fine and reach RUNNING state. The fix is mode=1777 on /var/run (matching /tmp); deferred to a follow-up to keep this PR focused.

gosu capabilities under rootless podman

The image's gosu binary is granted cap_setuid,cap_setgid+ep via setcap in the production stage. Without this, gosu deeptutor supervisord fails with operation not permitted inside the rootless podman user namespace (userns_mode: keep-id); with it, supervisord drops cleanly to UID 1000. Requires libcap2-bin in the image (one extra apt package, ~80 KB).

@enihcam
enihcam force-pushed the feature/rootless-podman-hardening branch from 34cd042 to 99af4c0 Compare June 17, 2026 13:22
enihcam added 4 commits June 17, 2026 21:40
The frontend no longer bakes the backend URL into the bundle at build
time and mutates it with sed on every container start. Instead:

  * web/lib/api.ts: apiUrl and wsUrl become one-line pass-throughs, so
    the browser fetches relative paths through the frontend
    (:3782/api/...). All existing call sites keep working unchanged.
  * web/proxy.ts: catches /api/* and /ws/* and rewrites them to
    DEEPTUTOR_API_BASE_URL at request time, reading
    DEEPTUTOR_AUTH_ENABLED for the auth-redirect check.
  * web/lib/auth.ts: switches from NEXT_PUBLIC_AUTH_ENABLED (build-time)
    to DEEPTUTOR_AUTH_ENABLED (set by the entrypoint on every start).
  * Dockerfile: drops the .env.local placeholder block, the sed -i in
    start-frontend.sh, and the runtime API-base fallback chain. Adds
    gosu, bakes a non-root 'deeptutor' user (UID 1000), and has the
    entrypoint chown /app/data and export DEEPTUTOR_API_BASE_URL and
    DEEPTUTOR_AUTH_ENABLED from data/user/settings/*.json on every
    start. supervisord is now invoked under 'gosu deeptutor'.
  * web/tests/api-resolve-base.test.ts: deleted (its 8 tests reference
    resolveBase / isApiBasePlaceholder, both gone; apiUrl/wsUrl are
    now pass-throughs so the rest is meaningless). The auth-redirect
    test is updated to prime DEEPTUTOR_AUTH_ENABLED instead.
The upstream ghcr.io/muchobien/pocketbase image's entrypoint uses
absolute paths (no /pb/ prefix): --dir=/pb_data --publicDir=/pb_public
--hooksDir=/pb_hooks. The previous example mounted at /pb/pb_data and
crashed on first start with 'mkdir /pb_data: read-only file system'.
Split the single mount into three matching the upstream entrypoint:
/pb_data, /pb_public, /pb_hooks.
Tested rootless-podman deployment shape:

  * read_only: true on every service
  * userns_mode: keep-id (host UID 1000 -> container UID 1000)
  * tmpfs mounts for /tmp, /run, /var/run, /var/log, /root, /home
  * bind mounts on ./data (no named volumes: podman auto-creates
    them with the userns-mapped root UID 100000, which 755's
    incorrectly against the host)
  * loopback-only port bindings (drop the 127.0.0.1: prefix to
    expose on all interfaces)
  * PocketBase sidecar uses the corrected /pb_data, /pb_public,
    /pb_hooks mount paths from the docker-compose fix.

The entrypoint chowns /app/data and gosu-drops to the unprivileged
deeptutor user (UID 1000) before starting supervisord, so under
userns_mode: keep-id the running process is your host user.

.env.example carries only the host-side loopback bindings and TZ. URL
knowledge lives in data/user/settings/system.json (read by the
entrypoint on every start -> DEEPTUTOR_API_BASE_URL -> proxy.ts). No
compose env var for the API base.
CONTAINERIZATION.md (7 sections):
  * Overview -- the architectural change (URL knowledge moves from
    build-time sed to request-time proxy rewriter) and the three
    deployment shapes that follow.
  * Docker (default) -- docker run with -p 3782/8001 and a
    deeptutor-data volume; remote / reverse-proxy deployment via
    system.json; host LLM providers (Ollama / LM Studio / etc.)
    via host.docker.internal.
  * Podman / rootless / read-only rootfs -- the compose.yaml walk-
    through, why each tmpfs/keep-id/read-only line is there, and a
    portable podman run equivalent. Notes the known
    /var/run/supervisord.pid follow-up.
  * Runtime configuration -- the data/user/settings/*.json layout
    and the no-compose-env-vars rule.
  * PocketBase -- the corrected mount path, single-user caveat.
  * Troubleshooting -- the four most likely first-boot issues.
  * Security notes -- non-root user, RO rootfs, keep-id, sandbox
    runner trade-off, CORS, auth gating.

README.md gets a one-line pointer in the 'Option 3 -- Docker'
section, right after the GHCR image list.
@enihcam
enihcam force-pushed the feature/rootless-podman-hardening branch from 99af4c0 to cd26210 Compare June 17, 2026 13:41
@enihcam

enihcam commented Jun 20, 2026

Copy link
Copy Markdown
Contributor Author

@pancacake Please take a look. The current version is not containerization friendly.

@pancacake

Copy link
Copy Markdown
Collaborator

@pancacake Please take a look. The current version is not containerization friendly.

Thanks for inviting. I've actually took a look days before, I think im gonna do some improvement on your pr before merging it. Thanks for your contribution!

pancacake and others added 4 commits June 20, 2026 20:56
The PR's relative-path + proxy.ts rewrite design depends on two env vars
(DEEPTUTOR_API_BASE_URL, DEEPTUTOR_AUTH_ENABLED) read server-side by the
Next.js middleware. As submitted they were only set by the Docker entrypoint,
and the auth one read a non-existent auth.json key — so auth enforcement was
silently disabled and the PyPI `deeptutor start` path was unwired.

- Export both vars from render_environment() (the single JSON-backed source of
  truth), so the Docker entrypoint AND the `deeptutor start` launcher get them
  consistently. DEEPTUTOR_API_BASE_URL falls back to localhost:<backend_port>.
- launcher.py: set both in common_env using the resolved backend_url, so they
  track port-conflict reassignment.
- Dockerfile: drop the duplicated, buggy per-var heredocs (one read the wrong
  key `auth_enabled` instead of `enabled`, leaving the proxy auth gate always
  off); the export_runtime_settings_to_env eval now provides both. Add them to
  the env-unset list for hygiene.
- Update the two placeholder tests to assert the new proxy-based design instead
  of the removed build-time sed mechanism; add DEEPTUTOR_* coverage in
  test_runtime_settings.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
…ploads

Two gaps in the PR's frontend changes:

1. Client auth awareness was permanently false. api.ts/auth.ts read
   process.env.DEEPTUTOR_AUTH_ENABLED in browser code, but Next.js only inlines
   NEXT_PUBLIC_ vars into the bundle — so the Sign-out / Admin affordances never
   rendered and apiFetch's 401 -> /login redirect never fired when auth was on.
   Resolve auth state at runtime from the backend instead: fetchAuthStatus()
   records it via setRuntimeAuthEnabled(), and components observe it through the
   new useAuthStatus() hook. Works on Docker (read-only rootfs), PyPI, and dev
   with no build-time baking. (The server-side gate in proxy.ts is unaffected.)

2. KB document uploads >10MB were truncated. All /api/* now flows through the
   proxy, whose body buffer defaults to 10MB while the backend accepts 100MB
   (DocumentValidator.MAX_FILE_SIZE). Raise experimental.proxyClientMaxBodySize
   to 110MB so uploads aren't silently cut off.

Adds a regression test that a 401 does NOT redirect when auth is disabled.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
CONTAINERIZATION.md described the old browser-talks-to-:8001-directly model,
which contradicts the PR's own server-side proxy design. Clarify that the
browser only ever talks to the frontend origin (:3782); /api/* and /ws/* are
forwarded to the backend inside the container, so publishing :8001 is optional.
Rewrite the remote/reverse-proxy section (single-container needs no API base;
next_public_api_base is only for split deployments) and the troubleshooting note.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@pancacake

pancacake commented Jun 20, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the exceptionally well-documented issue and PR, @enihcam — the rootless-podman + read-only-rootfs hardening, the PocketBase mount fix, and the move from build-time sed to a request-time proxy are all solid, and the writeup made review easy. 🙏

We dug into the request-routing change end-to-end (including reading the installed Next 16 source) and confirmed the core design works: proxy.ts (the Next 16 middleware) does proxy both cross-origin HTTP rewrites and WebSocket upgrades in standalone mode, so the relative-path approach is sound. We pushed a few follow-up commits on top before merging:

  • fix(deploy) — wired DEEPTUTOR_API_BASE_URL / DEEPTUTOR_AUTH_ENABLED through render_environment() (the single JSON-backed exporter) so both the Docker entrypoint and the PyPI deeptutor start launcher set them consistently. The entrypoint heredoc was reading auth_enabled instead of the canonical enabled key, which left the proxy auth gate always-off; that's fixed and de-duplicated.
  • fix(web) — the client auth flag read a non-NEXT_PUBLIC_ env var, so in the browser it was always false (Sign-out/Admin hidden, no 401→/login). Now resolved at runtime from /api/v1/auth/status via a useAuthStatus() hook. Also raised experimental.proxyClientMaxBodySize to 110MB so KB uploads up to the backend's 100MB limit aren't truncated by the proxy buffer.
  • tests/docs — updated the placeholder tests to assert the new proxy design and corrected CONTAINERIZATION.md (the browser only needs :3782; :8001 is optional).

Full gates green: ruff ✓, pytest (2447) ✓, tsc/eslint/node tests ✓, next build ✓. Merging to dev. Thanks again for pushing DeepTutor's deployment story forward!

-- From my claude code :)

@pancacake
pancacake merged commit 7bbc4bb into HKUDS:dev Jun 20, 2026
pancacake added a commit that referenced this pull request Jun 20, 2026
Self-service /profile page (account info, icon/image avatar, sign out)
implementing the profile half of #553. Contributed by @IliaAvdeev.

Integration performed during merge onto current dev:
- Migrate ProfileLink and the profile page off the removed AUTH_ENABLED
  build-time export to runtime auth status (auth.ts was refactored on dev
  by #577). ProfileLink reads the full status via fetchAuthStatus (it needs
  username/avatar, which the useAuthStatus hook does not expose); the page
  gates on status.enabled.
- Resolve en/zh locale conflicts as a union of dev and PR keys.
- Honor EXIF orientation in createImageBitmap so the primary decode path
  matches the <img> fallback.
- Drop an unreachable AUTH_ENABLED render guard in the profile page.

Verified locally: tsc, eslint, ruff, web node tests (142), pytest
tests/multi_user (81 passed).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
pancacake added a commit that referenced this pull request Jun 21, 2026
#577 moved /api and /ws behind a request-time Next.js proxy (web/proxy.ts),
so the browser only talks to :3782 and the frontend forwards to the backend
server-side. Update the Docker quickstart accordingly:

- drop the :8001 host mapping from the default `docker run`; publishing it is
  now optional (direct API access only)
- replace the "map both ports / no in-container proxy" warning with the
  single-port reality
- rewrite the reverse-proxy guidance: single-container needs no API base (point
  the proxy at :3782); only split deployments set next_public_api_base, which is
  read server-side and never sent to the browser

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
pancacake added a commit that referenced this pull request Jun 23, 2026
fix: run supervisord as root so services start under rootful Docker

PR #577 dropped supervisord itself to UID 1000 via gosu; under a rootful
daemon (Docker Desktop / Docker Engine) the root-owned /dev/fd/1,2 and
pidfile become unopenable, so backend+frontend never spawn (EACCES). Run
supervisord as root and drop each child to deeptutor via per-program
user= instead — app processes stay non-root in both rootful Docker and
rootless podman.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
pancacake pushed a commit that referenced this pull request Jun 27, 2026
…ss keep-id

Removes `user=root` from the [supervisord] section in both production and
development supervisord configs, and rewrites the entrypoint comment to
explain why.

Background: under rootless podman + userns_mode: keep-id, container PID 1
runs as the host user (UID 1000) instead of root. The previous design
explicitly forced supervisord to drop privileges to root (`user=root`),
which works under rootful daemons (PID 1 is root, has CAP_SETUID) but
fails under rootless + keep-id — supervisord sees `user=root` in its
config, tries to setuid(0), lacks CAP_SETUID, and exits with
"Error: Can't drop privilege as nonroot user" (per supervisord
options.py: refuses to drop privileges when not running as root).

Fix: omit `user=` from the [supervisord] section so supervisord inherits
PID 1's UID (root in rootful; UID 1000 in rootless + keep-id). The
backend and frontend programs still drop to the unprivileged deeptutor
user via per-program `user=deeptutor` — setuid(1000) works because
either PID 1 is root (rootful) or PID 1 is already UID 1000 (rootless
keep-id, where the setuid is a no-op). /dev/fd/1,2 ownership matches
PID 1's UID in both runtimes, so supervisord's stdout/stderr writes
succeed without an explicit setuid dance.

Verified locally: clean heredoc indentation; supervisord config parses;
both [supervisord] sections confirmed without `user=root`.

Closes the rootless-podman + read-only-rootfs follow-up to PR #577
(reported by the user after #577 + #586 merged).
vaskoyudha added a commit to vaskoyudha/deeptutor-for-programmer-fork that referenced this pull request Jul 25, 2026
…e proxy

Harden the published image for rootless podman with a read-only rootfs, and
move URL/auth knowledge from build-time sed to a request-time proxy (web/proxy.ts).
Includes follow-up fixes: JSON-backed DEEPTUTOR_* env wiring for both Docker and
the PyPI launcher (correct auth.json key), runtime client auth resolution,
100MB-safe upload proxy buffer, updated tests, and corrected docs.

Closes HKUDS#576. Co-authored with @enihcam.
vaskoyudha added a commit to vaskoyudha/deeptutor-for-programmer-fork that referenced this pull request Jul 25, 2026
Self-service /profile page (account info, icon/image avatar, sign out)
implementing the profile half of HKUDS#553. Contributed by @IliaAvdeev.

Integration performed during merge onto current dev:
- Migrate ProfileLink and the profile page off the removed AUTH_ENABLED
  build-time export to runtime auth status (auth.ts was refactored on dev
  by HKUDS#577). ProfileLink reads the full status via fetchAuthStatus (it needs
  username/avatar, which the useAuthStatus hook does not expose); the page
  gates on status.enabled.
- Resolve en/zh locale conflicts as a union of dev and PR keys.
- Honor EXIF orientation in createImageBitmap so the primary decode path
  matches the <img> fallback.
- Drop an unreachable AUTH_ENABLED render guard in the profile page.

Verified locally: tsc, eslint, ruff, web node tests (142), pytest
tests/multi_user (81 passed).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
vaskoyudha added a commit to vaskoyudha/deeptutor-for-programmer-fork that referenced this pull request Jul 25, 2026
HKUDS#577 moved /api and /ws behind a request-time Next.js proxy (web/proxy.ts),
so the browser only talks to :3782 and the frontend forwards to the backend
server-side. Update the Docker quickstart accordingly:

- drop the :8001 host mapping from the default `docker run`; publishing it is
  now optional (direct API access only)
- replace the "map both ports / no in-container proxy" warning with the
  single-port reality
- rewrite the reverse-proxy guidance: single-container needs no API base (point
  the proxy at :3782); only split deployments set next_public_api_base, which is
  read server-side and never sent to the browser

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
vaskoyudha added a commit to vaskoyudha/deeptutor-for-programmer-fork that referenced this pull request Jul 25, 2026
…visord

fix: run supervisord as root so services start under rootful Docker

PR HKUDS#577 dropped supervisord itself to UID 1000 via gosu; under a rootful
daemon (Docker Desktop / Docker Engine) the root-owned /dev/fd/1,2 and
pidfile become unopenable, so backend+frontend never spawn (EACCES). Run
supervisord as root and drop each child to deeptutor via per-program
user= instead — app processes stay non-root in both rootful Docker and
rootless podman.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
vaskoyudha added a commit to vaskoyudha/deeptutor-for-programmer-fork that referenced this pull request Jul 25, 2026
…ss keep-id

Removes `user=root` from the [supervisord] section in both production and
development supervisord configs, and rewrites the entrypoint comment to
explain why.

Background: under rootless podman + userns_mode: keep-id, container PID 1
runs as the host user (UID 1000) instead of root. The previous design
explicitly forced supervisord to drop privileges to root (`user=root`),
which works under rootful daemons (PID 1 is root, has CAP_SETUID) but
fails under rootless + keep-id — supervisord sees `user=root` in its
config, tries to setuid(0), lacks CAP_SETUID, and exits with
"Error: Can't drop privilege as nonroot user" (per supervisord
options.py: refuses to drop privileges when not running as root).

Fix: omit `user=` from the [supervisord] section so supervisord inherits
PID 1's UID (root in rootful; UID 1000 in rootless + keep-id). The
backend and frontend programs still drop to the unprivileged deeptutor
user via per-program `user=deeptutor` — setuid(1000) works because
either PID 1 is root (rootful) or PID 1 is already UID 1000 (rootless
keep-id, where the setuid is a no-op). /dev/fd/1,2 ownership matches
PID 1's UID in both runtimes, so supervisord's stdout/stderr writes
succeed without an explicit setuid dance.

Verified locally: clean heredoc indentation; supervisord config parses;
both [supervisord] sections confirmed without `user=root`.

Closes the rootless-podman + read-only-rootfs follow-up to PR HKUDS#577
(reported by the user after HKUDS#577 + HKUDS#586 merged).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants