Skip to content

Memory-only provider keys: kernel keyring store/exec/revoke (Tavily, user decision 2026-09-26) - #325

Merged
seathatflowsinourveins merged 5 commits into
mainfrom
claude/kernel-keyring-secrets-20260926
Sep 26, 2026
Merged

seathatflowsinourveins merged 5 commits into
mainfrom
claude/kernel-keyring-secrets-20260926

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator

Scope

  • Lane: lane:foundation, covering credentials and token practice.
  • User decision, 2026-09-26: "the key is fine for our practice, store it via SOTA env and future workflow seamlessly, just don't store them at file".
  • What this adds: a portable, memory-only way to keep a provider API key off disk on Linux hosts. Tavily is the first user.

What changes

  • scripts/kernel_keyring.py (stdlib, x86_64/aarch64 syscalls) with four commands:

    • store <name> reads the key from stdin. The key is created and permissioned in the process keyring, then moved into the kernel user keyring under a per-uid lock. --replace revokes the old key first.
    • status reports present or absent, never the value.
    • exec <name> <ENV_VAR> -- cmd sets the variable in that child's environment only. Only credential-shaped variable names are accepted, because LC_ALL, TERM, GIT_TRACE and the Python startup variables were shown to echo the value.
    • revoke revokes the key.

    No value is ever printed or put on argv. Nothing is written to disk. The key is gone after a kernel restart (wsl --shutdown or a Windows restart).

  • docs/secret-storage.md: a dated section, "Memory-only option: Linux kernel keyring". It covers:

    • private-input store commands with tracing disabled;
    • lifetime and scope;
    • the macOS equivalent, the native Keychain secret set/run path;
    • what it does not protect against: same-uid processes, and copies already in transcripts or prompt-capture stores.
  • recipes/tavily.md: on a keyring host, the standing instructions now point to kernel_keyring.py exec tavily_api_key TAVILY_API_KEY -- tvly …, and tvly login/init are forbidden there. It also records the first Research-endpoint use (the 2026-09-26 Alpaca landscape) without claiming qualification.

SOTA sources

Evidence-class table

Claim Class
store/status/exec/revoke, permissions, no value on stdout/stderr/argv, --replace, lock, allowlist synthetic unit tests (20), run on WSL2 kernel 6.18.33.2 x86_64
Live: the Tavily key is present in this host's keyring, and exec … tvly auth --json gives authenticated: true, method: env native, this host, value-free
The 26 documented commands pass secret_path_guard.check() local check
Mutation check: the tests fail against the prototype and 11 mutants local, not retained

Commands

uv run --no-project --with jsonschema --with pyyaml python -m unittest tests.test_kernel_keyring -v   # 20 OK
uv run ... -m unittest tests.test_secret_path_guard tests.test_credential_status ...                  # OK
python3 scripts/validate.py                                                                          # passed
python3 scripts/kernel_keyring.py status tavily_api_key                                              # present

Review

  • Two-family review (Claude Opus at max and GPT-6-Astra at max), then one repair round.
  • Fixed:
    • serialized store/revoke (concurrent-store race);
    • credential-only exec variables (the echo channels above);
    • hidden input with tracing disabled in the documented store command;
    • recipe instructions that pointed agents at the keyless or login path.
  • Follow-up (separate PR, owns the guard): scripts/hooks/secret_path_guard.py does not yet unwrap kernel_keyring.py exec … -- or block keyctl print|pipe|read, and TAVILY_API_KEY is not in its secret names. That PR also covers the inventory row, adoption/hooks/claude/SHA256SUMS, reinstalling the guard, and a keyring-backed tvly wrapper so upstream Tavily skills work without a file.
  • The pasted key was not rotated: the operator accepted this for practice use, and the docs describe rotation.

🤖 Generated with Claude Code

Scout and others added 3 commits September 26, 2026 00:25
The operator decided on 2026-09-26 that the Tavily API key must reach later
sessions and workflows without being stored in a file. Adds
scripts/kernel_keyring.py: the Linux kernel user keyring, a "user" key named
native-agent-stack:<name> with permissions 0x3F0B0000; the value comes from
stdin only, exec hands it to one child's environment only, and nothing prints
it.

Hardening over the session prototype:
- store creates the key in its own process keyring and links it into the
  user keyring afterwards, so the permission change also applies in a session
  whose keyring does not link the user keyring (systemd KeyringMode=private).
  The prototype ignored the failed SETPERM there, and a later exec failed with
  EACCES.
- Every keyctl result is checked; revoke falls back to invalidate when the
  session does not possess the key.
- store refuses an existing name unless --replace, which revokes it first.
- A terminal on stdin gets a prompt with echo off (the prototype echoed).
- Lowercase key names; [A-Z_][A-Z0-9_]* variables except loader/shell names;
  control characters refused; no argument or value in any message; no
  traceback; python -I re-exec.

tests/test_kernel_keyring.py (local integration, 14 tests, unique throwaway
names, own keyctl cleanup) covers the round trip, the child seeing the value
only in its environment, no value in any output, --replace, refusals, hidden
terminal input and private/linked session keyrings. It fails against the
prototype and against 11 hand-made mutants.

docs/secret-storage.md: dated section on the memory-only option (commands,
lifetime, scope, the macOS Keychain equivalent, limits). recipes/tavily.md:
dated keyring usage for search/extract/research; Research was first used on
2026-09-26 and its qualification receipt is pending.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…file Tavily auth

One bounded review round on the kernel keyring change:

- store checks again for an existing key after the value was read, under a
  per-uid lock (a Linux abstract socket name, not a file) that revoke also
  takes, so two stores of one name running at once cannot both succeed
  without --replace.
- exec sets only a credential variable: a name ending in KEY, KEY_ID, TOKEN,
  SECRET, PASSWORD or PASSPHRASE, never LD_*, DYLD_* or PYTHON*. Measured
  with a canary: PYTHONWARNINGS, PYTHONHOME, PYTHONIOENCODING (Python),
  LC_ALL (bash), TERM (tput) and GIT_TRACE (git) printed the value at
  startup, and PYTHONPYCACHEPREFIX made Python create directories named
  after it.
- Tests: a race held at the hidden prompt on a pseudo-terminal (plain and
  --replace), store and revoke waiting for the lock, the lock giving up, and
  the startup-variable refusals with a PYTHONWARNINGS positive control. Run
  against the previous script, 24 subtests fail and 3 error.
- docs/secret-storage.md: store through the hidden prompt first; the
  shell-variable form runs in a subshell with `set +x`, since bash xtrace
  printed the value; the guard's keyring gaps are stated as measured.
- recipes/tavily.md: the standing instructions point to the keyring path;
  no `tvly login`, and no `tvly init` without --skip-auth (both can write
  ~/.tavily/config.json).

Guard coverage of keyctl reads and of commands after `exec ... --` needs a
change that owns scripts/hooks/secret_path_guard.py; it is not in this commit.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 26, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@seathatflowsinourveins
seathatflowsinourveins enabled auto-merge (squash) September 26, 2026 04:26
Scout and others added 2 commits September 26, 2026 00:48
communicate() after closing Popen.stdin by hand raises 'flush of closed file' on
Python 3.12 (CI validate failure on #325); the host's 3.13 masked it. Reproduced
with uv --python 3.12 before the change (failures=1, errors=3) and OK after.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins
seathatflowsinourveins merged commit 9eece9d into main Sep 26, 2026
24 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/kernel-keyring-secrets-20260926 branch September 26, 2026 05:02
seathatflowsinourveins added a commit that referenced this pull request Sep 26, 2026
…lification receipt (follow-up to #325) (#329)

* Keyring guard, tvly-keyring wrapper and Tavily Research qualification receipt

Guard (scripts/hooks/secret_path_guard.py; SHA256SUMS updated):
- Unwrap `kernel_keyring.py exec <name> <ENV_VAR> -- <command>` (any path to
  the script, any launcher) and `tvly-keyring <args>`, so every existing rule
  applies to the command they start.
- keyring_variable_reference: the started command, code piped into it or a
  here-document names the injected variable beyond exec's own argument.
- environment_dump_in_keyring_exec: the started command dumps the environment
  it inherits (env, printenv, set, export -p, declare -p, ps e, ${!...},
  env or printenv as another program's argument, or whole-environment
  access in interpreter, awk or jq code).
- keyring_payload_read: keyctl print, pipe, read and dh_compute (keyctl(1):
  request, request2 and prequest2 print a key ID and pass), and payload reads
  in inline interpreter code (KEYCTL_READ, keyctl_read(, keyutils.read_key,
  kernel_keyring imports, raw keyctl syscalls with operation 11).
- `tvly auth` without --json (prints part of the key) is a
  native_token_print; ~/.tavily/config.json is a native store path;
  TAVILY_API_KEY is a secret name.

Credential inventory: `tavily` row with the new store kind kernel_keyring
(memory only, same-uid readable, lost at kernel restart). credential_status
reports it `unchecked` and never queries the keyring; set_credential refuses
it.

adoption/tools/tvly-keyring: POSIX sh wrapper that runs the real tvly through
kernel_keyring.py exec, never reads the key, passes arguments verbatim, and
exits 2 when the key is absent. Local integration tests with a fake tvly and
throwaway keyring keys (tests/test_tvly_keyring.py).

Evidence: tavily-research-qualification-20260926 receipt and a sanitized
summary of 33 completed Research (--model pro) reports: native provider
execution through the keyring exec, no quality comparison. The receipt and
the changed hash-listed files still need registration in
manifests/evidence.json.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Keyring guard repair: close the four reviewed bypasses at their root causes

Each reviewer finding was reproduced against the committed guard first
(check() returned None for all four), then fixed:

- Split-quoted exec argument (KK_DEMO_TO"KEN"): the variable check no
  longer subtracts a count of parsed arguments from raw-text matches. It
  exempts only the <ENV_VAR> argument's own span in the command after
  quote removal, so a split argument, an env/eval re-parse or a nested
  sh -c cannot use up the exemption. Backslash-newlines are joined first
  (a line continuation could hide the whole exec), and every raw-text
  rule also reads the unquoted command (sh -c 'echo $GH_TO''KEN').
- Launchers: strip_prefix reads launcher options as getopt does, from
  each tool's --help (stdbuf -o0, nice -n, timeout -s, sudo -iu, GNU
  time -f; xargs -i/-l/--max-lines take only glued values, checked with
  GNU findutils 4.9.0), and strips output redirections before a command.
  In a keyring exec, every argument naming a shell, interpreter, awk, jq,
  env, printenv, ps or a variable-printing builtin is read as a command
  from there on (watch, flock, taskset, find -exec).
- keyctl list/rlist print a user key's payload as integers (keyctl(1):
  "No attempt is made to check that the specified keyring is a keyring";
  act_keyctl_list/rlist in keyutils c076dff2). They pass only on @t @p @s
  @U @us @g, -1..-6 or %:name/%keyring:name; @A, serials and %user: are
  blocked, inline subprocess use included.
- Redirection operands (> >> 2> &> <<< < << <<-) are dropped before the
  tvly --json/--help and keyctl target checks.

The working-tree draft this replaces counted tokens across overlapping
segments and blocked the documented exec inside sh -c/bash -lc; those
forms are now ALLOWED regression cases.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Keyring guard: a URL argument is never a launched program (review low finding)

The Claude reviewer found the documented tvly-keyring extract form falsely blocked
(environment_dump_in_keyring_exec) when the URL's last path segment is env or printenv.
launched_commands now skips URL-shaped words; find -exec env and stdbuf /usr/bin/env
inside an exec stay blocked. Guard hash updated in adoption/hooks/claude/SHA256SUMS.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Keyring guard: close the GPT-6 re-check findings on the repair

Re-check of 74508203 (codex exec gpt-6-astra max, read-only): F2 resolved; F1, F3, F4
had variants and one false block. Each reproduced, then fixed:
- "$KK_DEMO_TOKEN"x: unquoted() merged it into another name; the variable rule now
  reads the written and the unquoted text (the exec argument exemption accepts quotes).
- keyctl rlist <serial> </dev/null: the serial was read as a descriptor; list and rlist
  are blocked whatever the target (kernel_keyring.py status answers presence).
- tvly auth {fd}>--json: bash's named descriptor is now a redirection.
- tvly-keyring search "export" ... --max-results: declare/export flags are short
  options only (false block removed).
Regression cases added; docs/secret-storage.md and the guard hash updated.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Register the keyring guard, tvly-keyring and Tavily Research receipt (hot-file commit)

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Re-register after rebasing onto #336 (hot-file commit)

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant