Persist gateway restart audit events#97189
Conversation
|
Codex review: needs maintainer review before merge. Reviewed July 13, 2026, 10:07 AM ET / 14:07 UTC. Summary PR surface: Source +636, Tests +655, Generated +38, Other -1. Total +1328 across 18 files. Reproducibility: not applicable. as a bug reproduction: current main demonstrably lacks durable restart-event persistence, and the contributor’s isolated SQLite proof verifies the proposed new capability. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Risk before merge
Maintainer options:
Next step before merge
Maintainer decision needed
Security Review detailsBest possible solution: Approve one explicit restart-audit contract—preferably aligned with the canonical audit ledger’s metadata-only, access, and retention semantics—then land the proven restart-critical ordering and redaction implementation. Do we have a high-confidence way to reproduce the issue? Not applicable as a bug reproduction: current main demonstrably lacks durable restart-event persistence, and the contributor’s isolated SQLite proof verifies the proposed new capability. Is this the best way to solve the issue? Unclear pending maintainer intent: the implementation is technically strong, but a separate restart ledger is not clearly better than extending or aligning with the canonical metadata-only audit contract. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 07d5a7e58ef6. Label changesLabel justifications:
Evidence reviewedPR surface: Source +636, Tests +655, Generated +38, Other -1. Total +1328 across 18 files. View PR surface stats
Security concerns:
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (24 earlier review cycles; latest 8 shown)
|
|
@clawsweeper re-review Behavior addressed: Gateway restart audit events now avoid blocking restart scheduling on the scheduled path, and restart warning fields no longer expose raw session keys. Findings addressed:
Real setup tested: Non-production isolated source checkout with a temporary Exact steps or command run after this patch:
Evidence after fix:
Observed result after fix: The restart request armed a pending restart, persisted a scheduled audit row in the isolated state database, and exposed only redacted session-key material in warning-format output. What was not tested: I did not run this against a live production Gateway or a real production restart; the runtime proof used an isolated temporary state directory by design. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Behavior addressed: Current head now redacts durable restart-audit storage, not just warning log output. Findings addressed:
Real setup tested: Non-production isolated source checkout with a temporary existing Exact steps or command run after this patch:
Evidence after fix:
Observed result after fix: Restart scheduling still arms and persists the audit row, while durable storage contains only redacted session/sensitive fields for the tested audit/preflight payload. What was not tested: I did not run this against a live production Gateway or a real production restart; the runtime proof used an isolated temporary state directory by design. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Behavior addressed: Current-head refresh after merging current Findings addressed:
Exact steps or command run after this patch:
Evidence after fix:
Observed result after fix: The restart-audit branch is updated to current What was not tested: I did not run this against a live production Gateway or a real production restart; the runtime proof used an isolated temporary state directory by design. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Behavior addressed: Current-head refresh after merging latest Findings addressed:
Exact steps or command run after this patch:
Evidence after fix:
Observed result after fix: The restart-audit branch is updated to latest What was not tested: I did not run this against a live production Gateway or a real production restart; the runtime proof used an isolated temporary state directory by design. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Current head follow-up after the latest CI failure:
Validation run after this patch:
Evidence after fix:
Note: I attempted the exact focused Vitest shard locally, but this environment got stuck in test transform. The pushed current head is intended to let CI verify that exact shard. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Behavior addressed:
Real setup tested: isolated source checkout on PR branch at Exact steps or command run after this patch:
Evidence after fix:
Observed result after fix:
What was not tested:
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
Dependency graph guard clearedThis PR no longer has blocked dependency graph changes. A future dependency graph change requires a fresh
|
|
@clawsweeper re-review Current head update after the watcher/manual refresh found a real shrinkwrap red. Head: 807ca8c Fix:
Validation:
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Current-head update after the dependency guard result: The previous shrinkwrap refresh was the wrong shape for this branch because it introduced lockfile-only churn and correctly tripped the dependency guard. I replaced that with a narrower source-side fix instead. What changed:
Validation on the new head:
Reviewer note: the current PR diff no longer contains lockfile changes; the dependency guard should no longer require a dependency-change override for this branch. |
|
@clawsweeper re-review Current-head update after refreshing this branch onto latest main. Head: What changed:
Validation on the new head:
Reviewer note: the prior dependency-guard blocker is no longer present; this head has no lockfile changes from the PR. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Current-head update after refreshing this branch onto latest Head: What changed:
Validation on the new head:
Reviewer note: the exact previous head had green CI, but GitHub showed the branch behind current |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Behavior addressed:
Findings addressed:
Exact steps or command run:
Evidence after fix:
Observed result after fix:
What was not tested:
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Behavior addressed:
Findings addressed:
Exact steps or command run:
Evidence after fix:
Observed result after fix:
What was not tested:
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Updated the current head ( Root-cause seam: current Behavior addressed: restart audit persistence remains ordered after restart-critical state transitions without bypassing the new signal-admission fence. Real setup tested: current PR checkout after merging the latest Exact steps or command run after this patch:
Evidence after fix:
Observed result after fix: the branch merges cleanly with current What was not tested: no live production gateway restart was performed; this is isolated code-path and test-suite proof. |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
@clawsweeper re-review Current-head update for Behavior addressed: Exact validation performed after this patch:
Observed result after fix:
What was not tested: |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Current-head follow-up for Delta since the previous proof:
Exact base/candidate discrimination for the remaining aggregate CI output:
No live gateway or production state was used. The restart-audit behavior and retention decision remain unchanged from the full proof above. |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
@clawsweeper re-review Current-head update for Fresh CI failures addressed:
Current-head validation:
The Node 24 test runtime was an isolated, checksum-verified official binary. No live gateway or production state was used. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@clawsweeper re-review Current-head update for The latest-main conflict in restart emission was resolved by preserving main's prepared-hook emission owner while continuing to pass the durable audit event through both the owner and signal-admission paths. No API surface was expanded. Current-head validation:
Validation ran in isolated systemd scopes under Node 24.18.0. No live gateway or production state was used. |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
@clawsweeper re-review Current-head follow-up for The prior aggregate reds were base-owned: a Control UI test-type regression and stale dead-export state. This head was uplifted onto main after those fixes landed; the restart-audit candidate did not absorb either unrelated change. Current-head validation after the uplift:
The base-only uplift did not change the restart-audit design or public surface. No live gateway or production state was used. |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
Closing this voluntarily. Since this PR opened, #98704 landed the canonical versioned, metadata-only audit ledger. This branch still creates a separate 512-row restart store with its own retention and privacy boundary. Re-expressing the change as queryable operator history through the canonical ledger would require a new public restart event family across the audit types, store/parser, RPC/CLI schemas, documentation, and generated clients. Reliable process-exit attribution would also need explicit durability semantics because the canonical audit writer is best-effort and background-backed. That is a broader product and security decision, and would make this PR larger rather than more focused. Current main also has durable restart intent/handoff/sentinel state and restart lifecycle logging, so the remaining unique gap is narrower: queryable historical restart attribution. The implementation in this branch is therefore no longer the right review surface. I would revisit the remaining gap only if maintainers explicitly want a minimal canonical |
Summary
gateway_restart_auditrows for restart scheduling/coalescing/rescheduling/deferral-bypass/emission events/restart, gateway restart tool, config writes, update runs, restart request API, and safe restart coordinationcron.isolated_agent_setup_timeoutcan be attributed from state instead of inferred from timingWhat Problem This Solves
Gateway restart forensics currently loses the initiating reason/source for internal safe restart requests. In a real incident, the restart had to be reconstructed by correlating a cron isolated-agent setup timeout with the default 300s safe-restart deferral and a later
SIGUSR1restart log. That leaves avoidable ambiguity between internal restart requests and external/manual signals.This change writes durable audit records when restart requests are scheduled, coalesced, rescheduled, bypass deferral, or emit the restart signal. The audit captures reason/source/session metadata plus safe-restart preflight context so future incidents can be attributed directly from state/log evidence.
Evidence
Local validation run under guarded
openclaw-heavy-run:pnpm format:check src/agents/tools/gateway-tool.ts src/auto-reply/reply/commands-session.ts src/gateway/server-methods/config-write-flow.ts src/gateway/server-methods/restart.test.ts src/gateway/server-methods/restart.ts src/gateway/server-methods/update.ts src/infra/restart-coordinator.test.ts src/infra/restart-coordinator.ts src/infra/restart.ts src/state/openclaw-state-db.generated.d.ts src/state/openclaw-state-schema.generated.ts src/state/openclaw-state-schema.sqlpnpm exec oxlint src/agents/tools/gateway-tool.ts src/auto-reply/reply/commands-session.ts src/gateway/server-methods/config-write-flow.ts src/gateway/server-methods/restart.test.ts src/gateway/server-methods/restart.ts src/gateway/server-methods/update.ts src/infra/restart-coordinator.test.ts src/infra/restart-coordinator.ts src/infra/restart.tspnpm check:architecturepnpm tsgo:corepnpm tsgo:core:testnode scripts/run-vitest.mjs run --config test/vitest/vitest.infra.config.ts src/infra/infra-runtime.test.ts --maxWorkers=1— 36 tests passednode scripts/run-vitest.mjs run --config test/vitest/vitest.infra.config.ts src/infra/restart.test.ts src/infra/restart-coordinator.test.ts --maxWorkers=1— 13 tests passednode scripts/run-vitest.mjs run --config test/vitest/vitest.gateway-methods.config.ts src/gateway/server-methods/restart.test.ts --maxWorkers=1— 6 tests passednode scripts/run-vitest.mjs run --config test/vitest/vitest.gateway-server.config.ts src/gateway/server-cron.test.ts src/gateway/server-restart-deferral.test.ts --maxWorkers=1— 28 tests passednode scripts/run-vitest.mjs run --config test/vitest/vitest.agents-tools.config.ts src/agents/tools/gateway-tool.test.ts src/agents/openclaw-gateway-tool.test.ts --maxWorkers=1— 14 tests passednode scripts/run-vitest.mjs run --config test/vitest/vitest.auto-reply-reply.config.ts src/auto-reply/reply/commands-session-restart.test.ts --maxWorkers=1— 6 tests passedAdditional fix after first CI pass:
pnpm db:kysely:genpnpm check:architecturenow pass locally