Skip to content

fix: protect bootstrap files during memory flush#38574

Merged
frankekn merged 7 commits intoopenclaw:mainfrom
frankekn:fix/38491-memory-flush-bootstrap-guard
Mar 10, 2026
Merged

fix: protect bootstrap files during memory flush#38574
frankekn merged 7 commits intoopenclaw:mainfrom
frankekn:fix/38491-memory-flush-bootstrap-guard

Conversation

@frankekn
Copy link
Copy Markdown
Contributor

@frankekn frankekn commented Mar 7, 2026

Summary

Describe the problem and fix in 2–5 bullets:

  • Problem: pre-compaction memory flush could overwrite workspace bootstrap files like MEMORY.md because the flush instructions only said to store durable memories.
  • Why it matters: a single flush turn could destroy long-lived workspace guidance and force manual git recovery before compaction completes.
  • What changed: memory flush now always injects guardrails that restrict writes to memory/YYYY-MM-DD.md, require append-only behavior, and mark MEMORY.md / SOUL.md / TOOLS.md / AGENTS.md as read-only, even when a custom flush prompt or system prompt is configured.
  • What did NOT change (scope boundary): this PR does not add a tool-layer writable-path sandbox; it hardens the prompt/system-prompt contract for the flush turn.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

Pre-compaction memory flushes now explicitly tell the model to write only to memory/YYYY-MM-DD.md, append instead of overwrite, and treat injected workspace bootstrap files as read-only. Custom memory-flush prompts keep the same safety guardrails.

Security Impact (required)

  • New permissions/capabilities? (Yes/No): No
  • Secrets/tokens handling changed? (Yes/No): No
  • New/changed network calls? (Yes/No): No
  • Command/tool execution surface changed? (Yes/No): No
  • Data access scope changed? (Yes/No): No
  • If any Yes, explain risk + mitigation:

Repro + Verification

Environment

  • OS: Darwin 25.3.0
  • Runtime/container: Node v22.18.0, pnpm 10.23.0
  • Model/provider: N/A for local verification (prompt construction + agent-runner tests)
  • Integration/channel (if any): N/A
  • Relevant config (redacted): default memory flush settings plus custom prompt override coverage in tests

Steps

  1. Reproduce the missing guard by inspecting DEFAULT_MEMORY_FLUSH_PROMPT and DEFAULT_MEMORY_FLUSH_SYSTEM_PROMPT in src/auto-reply/reply/memory-flush.ts.
  2. Trigger the memory-flush code paths through the focused unit/e2e suites.
  3. Run the repo local gate with pnpm check on the clean branch.

Expected

  • Memory flush prompt/system prompt should tell the model not to overwrite workspace bootstrap files.
  • Custom flush prompt overrides should still inherit the same safety guardrails.
  • Focused tests and pnpm check should pass locally.

Actual

  • The new guardrails are present in both default and custom flush prompts.
  • Focused tests pass.
  • pnpm check passes after syncing dependencies on the new origin/main base.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: default memory flush settings include the read-only bootstrap-file guard; custom flush prompt/systemPrompt overrides still include it; memory-flush agent-runner paths pass on the clean branch.
  • Edge cases checked: date placeholder still resolves to the canonical memory/YYYY-MM-DD.md path; upstream main's canonical-filename warning stays intact after cherry-picking.
  • What you did not verify: no end-to-end live model run; no tool-layer writable-path sandbox was added in this PR.

Compatibility / Migration

  • Backward compatible? (Yes/No): Yes
  • Config/env changes? (Yes/No): No
  • Migration needed? (Yes/No): No
  • If yes, exact upgrade steps:

Failure Recovery (if this breaks)

  • How to disable/revert this change quickly: revert this PR or override the memory flush prompt/system prompt in config while debugging.
  • Files/config to restore: src/auto-reply/reply/memory-flush.ts
  • Known bad symptoms reviewers should watch for: custom memory-flush prompt overrides losing the injected safety hints; runtime prompts using non-canonical memory filenames.

Risks and Mitigations

List only real risks for this PR. Add/remove entries as needed. If none, write None.

  • Risk: prompt-level guardrails still rely on the model obeying instructions.
    • Mitigation: the guardrails are now injected for both default and custom flush prompts, and focused tests lock in that contract. A future follow-up can add a tool-layer writable-path allowlist if stricter enforcement is needed.

@aisle-research-bot
Copy link
Copy Markdown

aisle-research-bot bot commented Mar 7, 2026

🔒 Aisle Security Analysis

We found 1 potential security issue(s) in this PR:

# Severity Title
1 🔵 Low Sandbox memory flush “append-only” implemented as non-atomic read-then-rewrite (lost updates / overwrite risk)

1. 🔵 Sandbox memory flush “append-only” implemented as non-atomic read-then-rewrite (lost updates / overwrite risk)

Property Value
Severity Low
CWE CWE-362
Location src/agents/pi-tools.read.ts:483-507

Description

The sandbox-mode implementation of appendMemoryFlushContent() does not perform a true append. Instead it:

  • Reads the full existing file content into memory (readOptionalUtf8File())
  • Concatenates the new content in-process
  • Calls sandbox.bridge.writeFile() to rewrite the entire file

This creates a race window and breaks the intended append-only guarantee:

  • Lost updates / clobbering: concurrent writers (another agent, user process, or another flush) can modify/create the file between the read and the final write, and their changes will be overwritten by the stale existing snapshot.
  • TOCTOU: stat() is used as an existence check before readFile(), and if the file is created after stat() returns null, the code will treat it as empty and then rewrite it.
  • DoS risk: reading the entire existing file into memory scales with file size; an unexpectedly large memory/YYYY-MM-DD.md can cause high memory usage.

Vulnerable code:

const existing = await readOptionalUtf8File({ ... });
const next = `${existing}${separator}${params.content}`;
await params.sandbox.bridge.writeFile({ filePath: params.relativePath, data: next, ... });

Recommendation

Implement a real append primitive for sandbox mode rather than read+rewrite.

Recommended approach:

  1. Add appendFile (or openAppend) to SandboxFsBridge that:
    • Validates path confinement (same boundary checks as writeFile)
    • Opens the destination with O_APPEND and O_NOFOLLOW (and rejects hardlinks) on the host path (similar to host-side appendFileWithinRoot)
    • Writes only the new bytes (optionally checking last byte for newline)

Example sketch:

// in SandboxFsBridge
async appendFile(params: { filePath: string; cwd?: string; data: Buffer | string }) {
  const target = this.resolveResolvedPath(params);
  this.ensureWriteAccess(target, "append files");
  await this.pathGuard.assertPathSafety(target, { action: "append files", requireWritable: true });

  const fd = fs.openSync(target.hostPath,
    fs.constants.O_WRONLY | fs.constants.O_APPEND | fs.constants.O_NOFOLLOW);
  try {
    fs.writeSync(fd, Buffer.isBuffer(params.data) ? params.data : Buffer.from(params.data, "utf8"));
  } finally {
    fs.closeSync(fd);
  }
}
  1. Update appendMemoryFlushContent() sandbox branch to call bridge.appendFile() and avoid reading the entire file.

If an append primitive cannot be added immediately, a mitigation is to do a compare-and-swap style write (e.g., verify mtime/size after read before commit), but true append is preferable.


Analyzed PR: #38574 at commit a0b9a02

Last updated on: 2026-03-10T05:29:11Z

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 7, 2026

Greptile Summary

This PR hardens the memory-flush feature by injecting three prompt-level guardrails — a target-path hint, an append-only hint, and a read-only hint for workspace bootstrap files — into both the default prompt/systemPrompt and any custom overrides supplied via config. The new ensureMemoryFlushSafetyHints() helper is applied in resolveMemoryFlushSettings before ensureNoReplyHint, ensuring the contract holds regardless of whether a user has customised the flush prompts.

Key changes:

  • Three new exported-private constants (MEMORY_FLUSH_TARGET_HINT, MEMORY_FLUSH_APPEND_ONLY_HINT, MEMORY_FLUSH_READ_ONLY_HINT) centralise the guardrail wording so it can't drift between prompt and system-prompt.
  • ensureMemoryFlushSafetyHints() iterates MEMORY_FLUSH_REQUIRED_HINTS and appends any hint not already present — a no-op for the defaults, which already embed all hints.
  • Unit and e2e tests are updated to assert the hints appear in both prompt and extraSystemPrompt for default and custom-override paths.
  • The YYYY-MM-DD placeholder in the user-facing prompt is correctly resolved to an actual date by the pre-existing resolveMemoryFlushPromptForRun; the system-prompt retains the literal placeholder, consistent with its role as format guidance.

Minor observation (not a bug): the ordering of hints in MEMORY_FLUSH_REQUIRED_HINTS (TARGET → APPEND_ONLY → READ_ONLY) differs from the ordering in DEFAULT_MEMORY_FLUSH_PROMPT (TARGET → READ_ONLY → APPEND_ONLY). Because ensureMemoryFlushSafetyHints uses substring includes checks rather than ordering, this has no functional impact — but aligning the array order to match the default-prompt order would remove a small source of confusion for future maintainers.

Confidence Score: 5/5

  • This PR is safe to merge — it only hardens prompt text and adds test coverage with no behaviour changes outside the flush-prompt contract.
  • Changes are narrowly scoped to prompt-string construction in memory-flush.ts. The new ensureMemoryFlushSafetyHints helper is straightforward, purely additive, and exercised by both unit and e2e tests. No data paths, network calls, or tool-execution surfaces are touched. Backward compatibility is preserved: existing default prompts already contain all three hints, so the helper is a no-op for them; custom-override prompts get the hints appended, which is the intended fix.
  • No files require special attention.

Last reviewed commit: 8223806

@openclaw-barnacle openclaw-barnacle bot added agents Agent runtime and tooling size: M and removed size: S labels Mar 7, 2026
@frankekn
Copy link
Copy Markdown
Contributor Author

frankekn commented Mar 7, 2026

Follow-up for the Aisle security finding: this no longer relies on prompt hints alone.

This branch now enforces memory-flush writes server-side:

  • trigger: "memory" runs force filesystem writes to stay under the workspace root.
  • Memory flush now receives a single canonical write target: memory/YYYY-MM-DD.md.
  • The generic write tool is wrapped to allow append-only writes to that one path.
  • edit and apply_patch are removed from memory-triggered runs.

Local verification I ran before pushing this update:

  • pnpm vitest run src/agents/pi-tools.workspace-only-false.test.ts src/auto-reply/reply/memory-flush.test.ts src/auto-reply/reply/reply-state.test.ts
  • pnpm exec vitest run --config vitest.e2e.config.ts src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts -t "memory flush"
  • pnpm check

@frankekn frankekn force-pushed the fix/38491-memory-flush-bootstrap-guard branch from 26a85a7 to 0be8e1d Compare March 10, 2026 04:32
@openclaw-barnacle openclaw-barnacle bot added the maintainer Maintainer-authored PR label Mar 10, 2026
@frankekn frankekn force-pushed the fix/38491-memory-flush-bootstrap-guard branch from 0be8e1d to a0b9a02 Compare March 10, 2026 04:37
@frankekn frankekn merged commit 96e4975 into openclaw:main Mar 10, 2026
28 checks passed
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a0b9a02e2e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +268 to +269
if (isMemoryFlushRun && !options?.memoryFlushWritePath) {
throw new Error("memoryFlushWritePath required for memory-triggered tool runs");
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Pass memory flush path to memory-triggered tool setup

Require this path only after it is fully plumbed: memory flush runs now call createOpenClawCodingTools with trigger: "memory", but runEmbeddedPiAgent does not forward memoryFlushWritePath into runEmbeddedAttempt (see src/agents/pi-embedded-runner/run.ts call around line 849), so this guard throws and aborts the flush turn whenever memory flush triggers. That means the new protection path can break the entire memory-flush workflow instead of restricting writes.

Useful? React with 👍 / 👎.

mukhtharcm pushed a commit to hnykda/openclaw that referenced this pull request Mar 10, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
aiwatching pushed a commit to aiwatching/openclaw that referenced this pull request Mar 10, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
Moshiii pushed a commit to Moshiii/openclaw that referenced this pull request Mar 11, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
Moshiii pushed a commit to Moshiii/openclaw that referenced this pull request Mar 11, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
dominicnunez pushed a commit to dominicnunez/openclaw that referenced this pull request Mar 11, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
dhoman pushed a commit to dhoman/chrono-claw that referenced this pull request Mar 11, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
Ruijie-Ysp pushed a commit to Ruijie-Ysp/clawdbot that referenced this pull request Mar 12, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
qipyle pushed a commit to qipyle/openclaw that referenced this pull request Mar 12, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
Interstellar-code pushed a commit to Interstellar-code/operator1 that referenced this pull request Mar 16, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn

(cherry picked from commit 96e4975)
senw-developers pushed a commit to senw-developers/va-openclaw that referenced this pull request Mar 17, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
V-Gutierrez pushed a commit to V-Gutierrez/openclaw-vendor that referenced this pull request Mar 17, 2026
Merged via squash.

Prepared head SHA: a0b9a02
Co-authored-by: frankekn <[email protected]>
Co-authored-by: frankekn <[email protected]>
Reviewed-by: @frankekn
alexey-pelykh pushed a commit to remoteclaw/remoteclaw that referenced this pull request Mar 22, 2026
alexey-pelykh pushed a commit to remoteclaw/remoteclaw that referenced this pull request Mar 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling maintainer Maintainer-authored PR size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pre-compaction 'write durable facts' overwrites workspace files (MEMORY.md destroyed)

1 participant