fix(config-audit): scrub pre-redactor argv values from historical config-audit.jsonl entries#80796
Conversation
|
Codex review: needs maintainer review before merge. Summary Reproducibility: yes. Source inspection shows current main only redacts newly captured argv fields and has no historical scrub path, and the supplied live logs show the PR detecting and scrubbing the historical residue. Real behavior proof Next step before merge Security Review detailsBest possible solution: Land this doctor-side historical scrub after normal maintainer review and green required checks, then let the linked bug close from the merged PR. Do we have a high-confidence way to reproduce the issue? Yes. Source inspection shows current main only redacts newly captured argv fields and has no historical scrub path, and the supplied live logs show the PR detecting and scrubbing the historical residue. Is this the best way to solve the issue? Yes. A What I checked:
Likely related people:
Remaining risk / open question:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 1ade7a53af1e. |
87d1d25 to
1139a1e
Compare
Verification: PR #80796 scrubs pre-redactor plaintext-token entries from
|
| Phase | Total entries | plaintext xoxb- |
plaintext xapp- |
redacted xoxb-1… |
redacted xapp-1… |
Config audit panel |
|---|---|---|---|---|---|---|
| BEFORE | 85 | 1 | 1 | 1 | 1 | — (panel didn't exist) |
| Dry-run | 85 (unchanged) | 1 (unchanged) | 1 (unchanged) | 1 | 1 | 2 entries … pre-redactor argv values. Run \openclaw doctor --fix`…` |
| --fix | 86 (+1, doctor's own audit entry) | 0 ✓ | 0 ✓ | 2 | 2 | Scrubbed 2 entries … pre-redactor argv values. Rotate any credentials… |
| Re-run (idempotency) | 86 (unchanged) | 0 (unchanged) | 0 (unchanged) | 2 | 2 | none ✓ — nothing left to scrub |
The +1 entry from --fix is the doctor's own config.write audit record for whatever migration it applied during the --fix pass (e.g. removal of agents.defaults.agentRuntime), redacted at write-time by the existing #75095 forward redactor. The +920 bytes is consistent with one new audit entry.
Dry-run preview panel (no --fix)
◇ Config audit ───────────────────────────────────────────────────────────╮
│ │
│ 2 entries in config-audit.jsonl still contain pre-redactor argv values │
│ (likely plaintext credentials at rest). Run `openclaw doctor --fix` to │
│ rewrite the argv/execArgv fields through the same redactor used for │
│ new entries. │
│ │
├──────────────────────────────────────────────────────────────────────────╯
File size before/after dry-run: identical (90727 bytes). Plaintext token counts unchanged. Confirms the dry-run path does not touch disk.
Actual scrub panel (--fix)
◇ Config audit ────────────────────────────────────────────────────────╮
│ │
│ Scrubbed 2 entries in config-audit.jsonl that still contained │
│ pre-redactor argv values. Rotate any credentials that may have been │
│ written to the log before the forward redactor shipped. │
│ │
├───────────────────────────────────────────────────────────────────────╯
The "Rotate any credentials" sentence is the right operational guidance — even after the at-rest redaction, the tokens were exposed to anyone who could have read the log between pre-#75095 write and the scrub.
Concrete entry transformation
The xoxb- entry from 2026-05-02T00:03:48.471Z (the exact entry called out in issue #80777):
BEFORE (from config-audit-BEFORE-2026-05-11T22-57-34Z.jsonl):
"argv": [
"/home/orin/.nvm/versions/node/v22.22.2/bin/node",
"/home/orin/Gittensor/Test/openclaw/openclaw.mjs",
"config",
"set",
"channels.slack.botToken",
"xoxb-<full-66-char-bot-token-bytes-in-plaintext>"
]AFTER (live file, post-scrub):
"argv": [
"/home/orin/.nvm/versions/node/v22.22.2/bin/node",
"/home/orin/Gittensor/Test/openclaw/openclaw.mjs",
"config",
"set",
"channels.slack.botToken",
"xoxb-1…03cT"
]All other record fields (ts, source, event, configPath, pid, ppid, cwd, execArgv, existsBefore, previousHash, nextHash, previousBytes, nextBytes, mode/inode metadata, suspicious, result) are preserved verbatim — forensic continuity intact.
Idempotency
Re-running pnpm openclaw doctor (no --fix) on the already-scrubbed log: no Config audit panel is emitted. The scrubber's same-input/no-rewrite path is correctly skipping already-redacted entries. Counter rewritten returned 0, so the dry-run preview correctly omits the note.
This matches the implementation's contract: redactConfigAuditArgv is a fixed point on already-redacted argv values (it passes *** and xoxb-1…03cT through unchanged), so re-applying the scrubber to a clean log is a no-op.
Safety verification (atomic write, no plaintext leak on failure)
The scrubber writes to a sibling *.scrub.tmp file at mode 0o600, then renames over the audit log; on any error it unlinks the temp file (src/config/io.audit.ts new scrubConfigAuditLog function). I did not crash-inject during this run, but the implementation matches the safety contract claimed in the PR body — a partial scrub cannot leave plaintext at rest because the rename is atomic and the temp file is owner-only-readable from creation.
No .scrub.tmp file is left behind after the successful runs:
$ ls ~/.openclaw/logs/*.scrub* 2>/dev/null || echo "no leftover scrub.tmp files"
no leftover scrub.tmp files
Files saved alongside this verification
config-audit-BEFORE-2026-05-11T22-57-34Z.jsonl— backup of the live audit log before any test ran (85 entries, 90727 bytes, mode0600).config-audit-AFTER-scrub-2026-05-11T23-10-21Z.jsonl— live audit log after the full scrub + idempotent re-run (86 entries, 91647 bytes, mode0600).doctor-DRYRUN-pr80796-2026-05-11T22-58-52Z.log— full doctor output without--fix, showing the preview panel and no disk write.doctor-FIX-pr80796-2026-05-11T23-06-12Z.log— full doctor output with--fix, showing the scrub panel and the actual disk rewrite.doctor-IDEMPOTENT-pr80796-2026-05-11T23-10-21Z.log— full doctor output on the already-scrubbed log, showing noConfig auditpanel.
Caveats
- Verified on Linux. The scrubber implementation is OS-agnostic (uses Node
fs.promiseswith explicitmode: 0o600), but a macOS/Windows live-run wasn't exercised in this verification. - The
--fixpass also applied at least one other migration during the same run (e.g.agents.defaults.agentRuntimeremoval — the bug from#80777's sibling issue at qa-reports/05-doctor-changes-past-tense/). That migration added the +1 entry in the AFTER count and is correctly redacted by the forward redactor on its way to disk. Net effect on the scrub verification: nothing — the 2 pre-redactor entries identified byDry-runwere both scrubbed, and the new entry written during--fixwas redacted at write-time. - The verification was done with the user's live config-audit.jsonl. Slack tokens at rest are now redacted-fingerprints rather than full bytes. Per the scrub panel's own guidance ("Rotate any credentials that may have been written to the log before the forward redactor shipped"), the user should still rotate the affected Slack bot/app tokens since they were exposed at rest in the log window pre-scrub.
|
Attached Real Behavior Proof with logs doctor-IDEMPOTENT-pr80796-2026-05-11T23-10-21Z.log |
1139a1e to
34ca817
Compare
34ca817 to
014d24a
Compare
014d24a to
d4c7785
Compare
…fig-audit.jsonl entries (openclaw#80777) PR openclaw#75095 added `redactConfigAuditArgv` so newly-written entries in `~/.openclaw/logs/config-audit.jsonl` mask token-shaped values, but the audit log has no rotation, retention, or one-shot scrub. Users who ran `openclaw config set <path> <secret-value>` between commit `748d6821d2` (audit added 2026-02-14) and `a853c5e8c2` (redactor merged 2026-05-01) still have plaintext Slack, Telegram, gateway, etc. tokens sitting in `config-audit.jsonl` at rest after upgrading. Add `scrubConfigAuditLog` in `src/config/io.audit.ts` that reads the audit log, rewrites every record's `argv` / `execArgv` fields through the existing forward redactor, and writes the file back atomically (`*.scrub.tmp` + rename, mode 0o600). Idempotent — already-masked entries pass through unchanged. Malformed lines are preserved verbatim and counted as `skipped` so the scrub never destroys forensic content it cannot understand. Concurrent-append guard: re-stat the audit log immediately before rename and abort (no on-disk change) if the file grew during the scrub, so a parallel appendConfigAuditRecord call is never overwritten by the rename swap. Wire into the doctor health flow as `runConfigAuditScrubHealth`: scan-only on a normal `openclaw doctor` (prints the count and suggests `--fix`); rewrite atomically on `openclaw doctor --fix` with a "rotate any credentials" note; print a "stop the gateway and rerun" hint on the abort path. No plaintext backup file is produced; file mode and parent dir mode are preserved. Per repo CLAUDE.md ("Legacy config repair: doctor/fix paths, not startup/load-time core migrations") the scrub is gated to `--fix` and runs only when the user opts in. Verified: - pnpm install --frozen-lockfile - pnpm test src/config/io.audit.test.ts - pnpm test src/commands/doctor-config-flow.test.ts - pnpm tsgo:core:test - pnpm exec oxfmt --check --threads=1 src/config/io.audit.ts src/config/io.audit.test.ts src/commands/doctor-config-audit-scrub.ts src/flows/doctor-health-contributions.ts Closes openclaw#80777
Signed-off-by: sallyom <[email protected]>
d4c7785 to
11bd19e
Compare
|
Maintainer verification before merge:
Maintainer-assisted follow-up commit adds the final stat guard after writing the temp scrub file, so a config-audit append between temp write and rename aborts without overwriting the live log. |
Summary
Describe the problem and fix in 2–5 bullets:
#75095addedredactConfigAuditArgvso newly-written entries in~/.openclaw/logs/config-audit.jsonlmask credential-bearing argv values, but the audit log is pure append-only with no rotation, retention, or scrub. Users who ranopenclaw config set <path> <secret-value>between2026-02-14(audit added) and2026-05-01(forward redactor merged) — ~2.5 months of releases — still have plaintext Slackxoxb-/xapp-, Telegram bot tokens, gateway tokens, etc. sitting inconfig-audit.jsonlat rest after upgrading. The file is mode0600so blast radius is owner-only, but the credentials are still live.#60826already flagged "credentials persist in the log file indefinitely unless manually scrubbed" and asked for a doctor-side scrub;#75095landed the forward write half but did not land the scrub. Reproduces deterministically on any host that went through the unredacted window.scrubConfigAuditLog(...)insrc/config/io.audit.ts— readsconfig-audit.jsonl, rewrites every record'sargv/execArgvfields through the existingredactConfigAuditArgvhelper, and writes the file back atomically (*.scrub.tmp+ rename, mode0o600). Idempotent — re-applying the redactor to already-masked entries is a no-op (the redactor passes***and--flag=***through unchanged), so subsequent doctor passes do not rewrite the file unless a genuinely unredacted record is still present. Concurrent-append guard: the function re-stats the audit log immediately before the rename; if the file grew between the initial read and the rename (anappendConfigAuditRecordcaller fired during the scrub), the function aborts without renaming and returnsaborted: true. No on-disk content is modified on abort, and the doctor surface prints a retry-when-idle hint. Wire it into the doctor health flow asrunConfigAuditScrubHealth(src/commands/doctor-config-audit-scrub.ts+ entry insrc/flows/doctor-health-contributions.ts): scan-only on a normalopenclaw doctor(printsN entries in config-audit.jsonl still contain pre-redactor argv values — run \openclaw doctor --fix` to scrub); rewrite atomically onopenclaw doctor --fix(printsScrubbed N entries; rotate any credentials that may have been written…`).redactConfigAuditArgvitself,snapshotConfigAuditProcessInfo,appendConfigAuditRecord{,Sync}, the audit log's mode (0o600) or parent directory mode (0o700), the on-disk JSONL shape (every field exceptargv/execArgvis preserved verbatim —ts,pid,ppid,cwd,previousHash,nextHash,suspicious,result, etc.). Malformed lines (parse failures, non-object payloads) are preserved verbatim and counted asskippedso the scrub never destroys forensic content it cannot understand. No plaintext backup file is ever produced; the*.scrub.tmpsibling is created at mode0o600and removed on any error path before re-raising.Change Type (select all)
Scope (select all touched areas)
Linked Issue/PR
#75095plaintext-token entries in~/.openclaw/logs/config-audit.jsonlare never scrubbed after upgrade — live Slack/Telegram/etc. credentials persist at rest indefinitely #80777#75095closed it on the forward-write half only)Real behavior proof (required for external PRs)
Behavior or issue addressed: pre-
#75095plaintext-token entries in~/.openclaw/logs/config-audit.jsonlare never scrubbed after upgrade; live Slack/Telegram/gateway tokens persist at rest indefinitely. See [Bug]: pre-#75095plaintext-token entries in~/.openclaw/logs/config-audit.jsonlare never scrubbed after upgrade — live Slack/Telegram/etc. credentials persist at rest indefinitely #80777.Real environment tested: Ubuntu 24.04 (Linux 6.8.0-110-generic). Local OpenClaw source checkout rebased onto upstream
eec0d16458(currentmain). The reporter's box has~/.openclaw/logs/config-audit.jsonlat mode0600with two unredacted Slack token entries written on2026-05-02by a pre-#75095build, plus a separate redactor-aware build at HEAD that produces correctly-masked new entries.Exact steps or command run after this patch (against a synthetic
OPENCLAW_STATE_DIRso the captures use known fake tokens instead of any live credentials on the reporter's box):pnpm install --frozen-lockfilepnpm build/tmp/openclaw-fix-80777/logs/config-audit.jsonlwith three synthetic records (two with pre-redactorxapp-1-FAKE-FOR-QA-PROOF-…/xoxb-FAKE-FOR-QA-PROOF-…argv values, one already-clean record withargv = ["…", "ui.theme", "dark"]).OPENCLAW_STATE_DIR=/tmp/openclaw-fix-80777 pnpm openclaw doctor --non-interactiveOPENCLAW_STATE_DIR=/tmp/openclaw-fix-80777 pnpm openclaw doctor --fix --non-interactivegrep -c -E 'xapp-1-FAKE|xoxb-FAKE' /tmp/openclaw-fix-80777/logs/config-audit.jsonlEvidence after fix:
Scan-only run (
openclaw doctor --non-interactive, no--fix),Config auditpanel emitted from real terminal capture:Repair run (
openclaw doctor --fix --non-interactive),Config auditpanel emitted from real terminal capture:Verbatim on-disk before/after of the fixture (the
then …lines below are descriptive separators between commands, not actual shell#comments — repaginated to avoid markdown's leading-#heading parser):The post-scrub file ends
…"xapp-1…6789"…and…"xoxb-F…cdef"…— the standard start/end-keep mask the redactor produces; every other field of every record is byte-identical; mode is preserved at0600.Targeted unit-test capture (real fs temp dirs via
createSuiteTempRootTracker):Pre-fix behaviour from the issue body (token bytes obfuscated to
Xby the original reporter; the original file on the reporter's box still has the live token bytes at rest):openclaw doctorsurfaces the scan-only note when any unredacted entry remains;openclaw doctor --fixrewrites the file atomically through the same redactor used for forward writes. The on-disk JSONL shape, file mode, and forensic metadata are preserved. Subsequent doctor passes on a clean log are silent.#75095plaintext-token entries in~/.openclaw/logs/config-audit.jsonlare never scrubbed after upgrade — live Slack/Telegram/etc. credentials persist at rest indefinitely #80777, taken on a realv2026.5.6host with live tokens still at rest.xoxb-real-bot-token-…,xapp-1-A1B2C3-…) that match the redactor'sxox[baprs]-…/xapp-…patterns. The reporter's box's live entries can be scrubbed by runningopenclaw doctor --fixonce after upgrading to this PR's tip.resolveConfigAuditLogPathwhich already handles platform differences; this PR adds no new platform-specific code.openclaw doctor --fix(single-process, user-initiated) and by writing to*.scrub.tmpthen renaming —appendFilecallers continue writing toconfig-audit.jsonl, and worst case a concurrent append is dropped onto the pre-rename file just before the rename swap (next doctor run would catch the still-unredacted residue).Root Cause (if applicable)
#75095(commita853c5e8c2, merged2026-05-01) added forward redaction by callingredactConfigAuditArgvfromsnapshotConfigAuditProcessInfo. The audit log was added earlier in commit748d6821d2(2026-02-14) as a pureappendFile/appendFileSyncpair (src/config/io.audit.ts:441-467) with no rotation, retention, migration, or scrub callers anywhere in the tree (grep -rn 'config-audit\|configAudit' src/shows zero rewrite paths). So writes during the ~2.5-month unredacted window persisted full argv at rest, and the forward-write fix did not retroactively clean them up.io.audit.tscovered only forward-write redaction and append shape, not historical-record rewrite. This PR adds both — the doctor health hook inrunConfigAuditScrubHealth, and the four-case unit-test coverage ofscrubConfigAuditLog.#60826already enumerated the at-rest residue in its third Impact bullet and offered a workaround script (scrub-config-audit-log.js).#75095addressed the forward-write half of the report (which is the easier half — at-write time the runtime has full context) but did not land the scrub script or any doctor-time scrub action. This PR finishes the report's other half via the maintainer-preferred path (doctor--fix, not startup migration).Regression Test Plan (if applicable)
src/config/io.audit.test.ts— five new cases added at the end of the existingdescribe("config io audit helpers", …)block, all using real fs via the existingcreateSuiteTempRootTrackerpattern (no fs mocking).config-audit.jsonlwith one pre-redactor record (argv ends inxoxb-real-bot-token-…) and one already-redacted record. AfterscrubConfigAuditLog, the first record's argv[5] no longer containsreal-bot-token, the other fields (ts,suspicious, etc.) are preserved verbatim, the second record is byte-identical to its pre-scrub form, and the file mode is0o600. A secondscrubConfigAuditLogcall returnsrewritten: 0, aborted: false.{ scanned: 0, rewritten: 0, skipped: 0, aborted: false }and does not create the file.skipped. A two-line file with one malformed line ({this is not valid json) followed by one valid unredacted record is rewritten such that the malformed line is byte-identical and the valid record's token is masked.dryRun: truedoes not write. A scrub of a file that would otherwise be rewritten leaves the file's bytes unchanged but returnsrewritten: 1, aborted: false, so the scan-only doctor branch can preview without mutating.statreturns a size 200 bytes larger than the actual file (simulating anappendConfigAuditRecordcall between the initial read and the rename),scrubConfigAuditLogreturnsaborted: true, the audit file's bytes are unchanged, and no*.scrub.tmpis left on disk.scrubConfigAuditLogdirectly against a real fs temp dir match the production code path; a seam or e2e test would add scaffolding without catching anything the unit cannot. The existingredactConfigAuditArgvis already covered by 11+ unit tests for forward-write semantics.io.audit.test.tscovered forward-write redaction and JSONL append shape, not historical-record rewrite.User-visible / Behavior Changes
openclaw doctor(no--fix): emits a singleConfig auditnote when one or more historical records still contain pre-redactor argv values, e.g.7 entries in config-audit.jsonl still contain pre-redactor argv values (likely plaintext credentials at rest). Run \openclaw doctor --fix` to rewrite the argv/execArgv fields through the same redactor used for new entries.` No file mutation. Silent when the audit log is clean or absent.openclaw doctor --fix: rewritesconfig-audit.jsonlatomically throughredactConfigAuditArgvand emitsScrubbed 7 entries in config-audit.jsonl that still contained pre-redactor argv values. Rotate any credentials that may have been written to the log before the forward redactor shipped.File mode is preserved (0o600); on-disk JSONL shape is preserved (onlyargv/execArgvfields are masked).--jsonoutput (openclaw doctor --jsonif used) is unchanged.Diagram (if applicable)
Security Impact (required)
NoYes— at-rest credentials inconfig-audit.jsonlhistorical entries are now masked byopenclaw doctor --fixvia the same redactor used for forward writes.NoNoNoYes, explain risk + mitigation:.bakfile at a wider permission, or a temp file left behind on crash).*.scrub.tmpcreated at mode0o600and renamed over the audit log on success. On any error path, the temp file is unlinked before the function re-raises (best-effort cleanup; the rename failure is the actionable error). No plaintext backup file is produced. The audit log's mode (0o600) and parent dir mode (0o700) are preserved. The note printed after a successful scrub explicitly tells the user to rotate any credentials that may have been written to the log before the forward redactor shipped, because mode-0o600-at-rest is not equivalent to never-disclosed (cron sweeps, backup tools, log shippers, etc.).Repro + Verification
Environment
eec0d16458(currentmain)openclaw config set <secret-path> <secret-value>during the unredacted windowSteps
~/.openclaw/logs/config-audit.jsonlwritten by a pre-#75095OpenClaw build (i.e. before commita853c5e8c2, merged2026-05-01), containing at least oneconfig.writerecord whoseargvends in a credential-bearing value.pnpm openclaw doctor --non-interactive.Config auditnote that surfaces the count of unredacted entries.pnpm openclaw doctor --fix --non-interactive.wc -l ~/.openclaw/logs/config-audit.jsonlis unchanged (or differs only by whether the file already had a trailing newline) andgrep -c -E 'xoxb-[A-Za-z0-9-]{20,}' ~/.openclaw/logs/config-audit.jsonlreturns0.Expected
N entries in config-audit.jsonl still contain pre-redactor argv values …note whereNmatches the actual count of unredacted entries.Scrubbed N entries …note; the file'sxoxb-/xapp-/ bot-token bytes are replaced with the redactor's mask (start-keep /…/ end-keep), but every other field of every record is byte-identical, and file mode remains0o600.Actual (after this patch — matches expected)
src/config/io.audit.test.ts(added in this PR) exercise exactly the steps above against real-fs temp dirs and pass green.Evidence
pnpm test src/config/io.audit.test.tsand the before-stategrepof a realconfig-audit.jsonlis in the Real-behavior-proof section above.Human Verification (required)
argv = [..., "xoxb-real-bot-token-…"]is rewritten such that argv[5] no longer contains the original token bytes; every other record field is byte-identical (new unit test 1).skipped(new unit test 3).dryRun: truedoes not write to disk (new unit test 4).0o600(new unit test 1,fs.statSync(auditPath).mode & 0o777).src/config/io.audit.test.tsstill pass (regression check).pnpm test src/commands/doctor-config-flow.test.ts→ 32/32 pass (no doctor-flow regressions from the new health contribution).argv— skip mutation for that record's argv field (silently preserved).openclaw doctor --fix(single-process, user-initiated), and the*.scrub.tmp+ rename pattern keeps the atomic guarantee.Review Conversations
Compatibility / Migration
YesNoNoopenclaw doctor --fix(per repo CLAUDE.md: "Legacy config repair: doctor/fix paths, not startup/load-time core migrations.").Risks and Mitigations
*.scrub.tmpat mode0o600, renamed over the audit log on success. On any error, the temp file is unlinked. No plaintext backup is produced. File mode and parent dir mode are preserved. The post-scrub note tells the user to rotate any credentials that may have been written before the redactor shipped — at-rest mode0o600is not equivalent to never-disclosed.config-audit.jsonlwhile the scrub is mid-rewrite. The append goes to the pre-rename file and would be lost by the rename swap.scrubConfigAuditLogre-stats the audit log immediately before the rename. Ifstat.size !== originalByteLength, the function aborts withaborted: trueand does not rename — no records are dropped. The doctor surface printsConfig audit scrub was aborted because new entries were appended to config-audit.jsonl during the rewrite. No records were modified. Stop the gateway (or wait until it is idle) and rerun \openclaw doctor --fix`.New unit test #5 (aborts without overwriting when the audit log was appended to mid-scrub) locks in the abort path against a regression. There is a microsecond-level race window between the pre-renamestatand therename` itself; this is the smallest racey window achievable without coordinated locking, and any append during that window would simply re-surface on the next doctor pass.scrubConfigAuditLogcall assertsrewritten: 0against the post-scrub file, locking in idempotency under whatever the current redactor patterns produce.