feat(crypto): generate the master key on first launch, never replacing an existing one - #109

Summary

A new deployment could not start until the operator had produced a master key by hand (openssl rand -base64 48 > docker/secrets/app_secret_key). Because Compose validates a file-backed secrets: entry before it reads the rest of the stack, a missing file failed docker compose up outright rather than producing a startup error anyone could act on — a first-run blocker, which a user hit. The key is now generated on first launch when none is supplied.

The convenience is the small part; the invariants are the substance, because a second master key silently orphans every field-encrypted secret (registry credentials, git tokens, the OIDC client secret, TOTP seeds, scheduled-backup passphrases) while the app looks perfectly healthy.

Precedence (was one source; generation is strictly last)

First match wins:

1. SCRYE_APP_SECRET_KEY_FILE (default /run/secrets/app_secret_key) — the Docker secret. Unchanged, still highest.
2. SCRYE_APP_SECRET_KEY_AUTOGEN_FILE (new, default /data/app_secret_key) — the key a previous start generated.
3. A key generated now, written to (2) — only when neither file exists and SCRYE_APP_SECRET_KEY_AUTOGENERATE (new, default true) is on.

No key is read from an env var or an image layer, as before. SCRYE_ALLOW_WEAK_MASTER_KEY remains a validation opt-out, not a key source. The KDF and token format are untouched, so existing ciphertext keeps decrypting (regression-tested), and existing deployments resolve their key exactly as they did.

Invariants enforced

• An existing key file is used, never replaced. Unreadable, empty, non-base64, too short or malformed → startup fails. Generation follows only from a proven absent file, never a failed load. Absence is proven too: only ENOENT/ENOTDIR count as absence, any other stat error refuses, so an unreadable parent directory can never read as "no key here".
• An explicitly configured path is an assertion. A set SCRYE_APP_SECRET_KEY_FILE with no file there refuses to start rather than substituting a generated key — on an existing deployment that state means an unmounted secret, not a fresh install.
• The two files may not disagree. A supplied secret alongside a previously generated key it does not cover refuses to start. Version-aware, not material-aware: stored tokens name their key version, so the same key under a different version number still would not be found at decrypt time.
• Concurrent starts cannot both generate. O_CREAT|O_EXCL; the loser reads the winner's key. A key file caught empty mid-write is retried — and only that case, via a dedicated MasterKeyFileEmptyError, so a malformed key still fails immediately.
• A generated key that fails its permission check is removed by the call that created it, so the next start cannot adopt the file this one refused. That is the only place a key file is ever deleted.

Generation itself: os.urandom(48) base64-encoded — the documented openssl rand -base64 48 form, clearing the SEC-3 entropy floor with no weak-key opt-out — written O_CREAT|O_EXCL + fsync, chmod 0600 (the O_CREAT mode is a umask-masked ceiling), then re-stat verified for mode 0600 and owner uid before use, plus one INFO line naming the path and warning that the file must be backed up.

What changed

• backend/app/core/crypto.py — resolve_master_keys() and the generation/verification/race machinery; MasterKeyFileEmptyError.
• backend/app/core/config.py — the two new settings and app_secret_key_file_is_explicit.
• backend/app/main.py — startup resolves the key (so the generation notice lands in the startup log) and reports its source.
• docker/docker-compose.yml, README paste-in stack — the secret is no longer required; the Docker-secret blocks are kept, commented out, for deployments that want the key and data on separate mounts.
• README.md — where the key lives, the precedence order, auto-generation, back it up, what is lost if you don't, expanded first-run troubleshooting, the config table, and the security-model bullet. Also documents the trade-off that the generated key sits on the same volume as the database it protects.
• CONTRIBUTING.md, .env.example (regenerated), CHANGELOG.md, docs/ROADMAP.md.
• CLAUDE.md § Hard security rules — amended in the same commit rather than left contradicting the code (the Docker secret file remains the recommended production mechanism and keeps its precedence).

One premise corrected

The scoping said a backup bundle without the master key is useless for the encrypted fields. That is not true of Scrye's bundles: §8's design re-wraps each secret under the user's passphrase, so a bundle restores onto a host with a different master key — and now onto a fresh deployment that generated its own. The master key is required for a volume/file-level backup, where the rows are master-key ciphertext, so README § Backup & restore draws that line explicitly instead.

Testing

• backend/tests/test_master_key_autogeneration.py — 35 cases: generation on a clean volume, entropy floor without the opt-out, mode 0600 and owner, the one-time INFO backup notice, existing keys reused verbatim, every malformed/unreadable/undeterminable case failing rather than regenerating, the explicit-but-missing refusal, the two-key interlock (including the same key under a different version and the accepted carry-forward), eight-thread concurrent startup producing exactly one key, permission-check cleanup, and a generated key still decrypting its data across a restart.
• Full suite green: 622 passed, 3 skipped on Python 3.14.6; ruff + black clean; .env.example in sync.
• Verified end-to-end against a running instance: first boot generated the key at 0600 and /healthz reported healthy, a registry secret was stored, and after a restart the same key was reused and a backup bundle built successfully — which decrypts every stored secret to re-wrap it. The image build / docker compose up could not be exercised in this environment (no Docker daemon available); CI's image job covers it.

Note that the concurrency test earned its place: it caught a real race where a third process that merely saw the path exist read the winner's zero-length file, and then a TOCTOU in the first fix for it. See docs/ARCHIVE.md § Deviations (2026-07-29) for the full rationale.