Skip to content

fix(settings): save builtin agent overrides atomically - #2627

Merged
nicobailon merged 4 commits into
nicobailon:mainfrom
quifox:fix/settings-atomic-save-main
Oct 2, 2026
Merged

nicobailon merged 4 commits into
nicobailon:mainfrom
quifox:fix/settings-atomic-save-main

Conversation

@quifox

@quifox quifox commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Builtin agent override saves previously truncated settings.json before writing the replacement, so an interrupted write could leave invalid JSON for the next session. The shared override writer now stages the complete JSON and atomically replaces the target using the existing atomic writer.

The settings format and trailing newline are retained. Existing mode bits are reapplied, final symlinks are followed through their physical parent, and an existing target's writability is checked before staging. This covers the four builtin override mutators; other settings writers are outside this change.

Atomic replacement requires a writable parent directory and replaces the inode. Hardlink aliases retain old contents; ownership, ACLs and extended attributes are not copied. This does not add concurrent-update serialization or fsync durability.

Validation: 170 focused settings/override/agent-management/profile tests and typecheck passed. The six new tests include real API process interruption, rename EIO, exact formatting, read-only rejection and symlink/mode controls; the base passes the behavior controls and fails both atomic-fault regressions. POSIX controls are conditional on platform; a Windows host was not tested.

Follow-up for f0eec3b8

Target resolution now follows native filesystem semantics, so directory symlinks are followed before .., including dangling final targets whose physical parent exists. Missing parents, missing directory targets with trailing separators, and link cycles are rejected without replacing the settings link.

With Node 24.19.0, the updated code passes 202 focused tests and typecheck on both Linux and macOS, with no skipped tests. The original PR production code fails 13 of the same 20 targeted settings tests; the repaired code passes all 20. The PR workflow includes Windows tests; POSIX link/mode controls are conditional on platform.

@quifox
quifox marked this pull request as ready for review October 1, 2026 16:25
@greptile-apps

greptile-apps Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] Changes how builtin agent settings are saved to disk.

The portable-tests requirement needs attention before merge; exact-head required CI has not been verified.

Findings

  1. P1 Read-only test depends on host ▶
Fix with agent prompt
### Issue 1
test/unit/settings-atomic-save.test.ts:343-345
This test gets a user identity from `/etc/passwd` when running as root, changes file ownership and modes, and expects a child process to receive `EACCES`. If the fixture cannot be prepared, the test is skipped, so the read-only behavior is not checked on that host. This violates the repository’s portable-tests directive to prefer deterministic filesystem failures over POSIX permission assumptions.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Summary

The PR stages builtin-agent override settings before atomically replacing the settings file, preserving its JSON formatting, trailing newline, and mode bits. The latest changes refine physical symlink-target resolution and add filesystem regressions.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Read settings] --> B[Apply builtin override]
  B --> C[Resolve physical target]
  C --> D[Write staged JSON]
  D --> E[Restore mode bits]
  E --> F[Rename over target]
Loading

Reviews (2) · Last reviewed commit: "fix(settings): resolve physical targets ..."

@nicobailon

Copy link
Copy Markdown
Owner

Thanks, the truncation bug is real, and moving all four builtin override mutators onto the shared atomic writer is the right shape. Review found one regression at 2cfb043.

A link target with .. through a directory symlink writes the wrong file. resolveSettingsWriteTarget (agents.ts:984) builds the target with path.resolve(realpath(dirname), readlink(target)). That collapses .. as text before the filesystem follows intermediate directory symlinks, so it isn't the file the kernel opens. Repro through saveBuiltinAgentOverride:

  • project/.pi/via → directory symlink to actual/nested
  • project/.pi/settings.json → symlink with text via/../target.json
  • actual/target.json = {"sentinel":"physical"}, project/.pi/target.json = {"sentinel":"decoy"}

Reading settings.json returns the physical file. After the save, actual/target.json is unchanged, and the unrelated project/.pi/target.json has been overwritten with the physical contents plus the override. The old writeFileSync(filePath) let the kernel resolve the path and wrote the right file.

Please resolve existing targets through the filesystem. On macOS with Node, fs.realpathSync(settingsPath) returned the decoy too, while fs.realpathSync.native(settingsPath) returned the correct file. Keep a narrow fallback for dangling links that doesn't collapse link-target components early, and add the repro above as a regression test. That also makes the manual 40-iteration loop unnecessary for existing targets.

The mode chmod before rename and the W_OK check are both needed. Keep them.

Optional test reductions:

  • The 40-link boundary test (settings-atomic-save.test.ts:205-242) encodes the loop's limit and skips on macOS. A short ordinary chain plus the regression above covers the contract.
  • The JSON.parse equality checks right after the exact-byte assertion repeat it, and the interrupted temp file's exact 12-byte length is an implementation detail.

Resolve existing targets with native realpath and follow dangling final
links only when their physical parent exists. Preserve link text so the
filesystem follows directory symlinks before parent traversal.

Reject missing directory targets with trailing separators. Cover target
selection and normalize fault injection paths for filesystem aliases.
Comment thread test/unit/settings-atomic-save.test.ts Resolved
@nicobailon
nicobailon merged commit 63ac9c2 into nicobailon:main Oct 2, 2026
14 checks passed
Markusli added a commit to Markusli/pi-subagents that referenced this pull request Oct 3, 2026
* feat: let hosts require child extensions for every runner (nicobailon#2643)

* feat: let hosts require child extensions for every runner

registerRequiredChildExtensions gains requireForAllRunners. When set,
the host's required extensions become mandatory for every launch rather
than being excluded on routes that cannot load them. Only a local native
Pi child loads these extensions, so a launch that resolves to an external
CLI runner, an external-job runner, or any machine placement is rejected
before a runner process is spawned, a provider job starts, or a task is
handed off. The flag is stamped onto each snapshot entry so it survives
the nested, detached, resumed, recovered, and appended carriers, and a
disposed registration or an extensions: [] override cannot drop it.
Existing registrations without the flag keep excluding external runners.

Related to nicobailon#2639

* fix: write the retained extension snapshot without an extra parent-session read

The chain launch read ctx.parentSessionId once more before building its
steps, so a launch whose parent identity changed between reads was no
longer rejected as inconsistent. Write the snapshot after the launch
parent is resolved and reuse that value.

Related to nicobailon#2639

* fix: name required extension IDs when a foreground machine launch is refused

A foreground native launch with machine placement was already refused
before remote work by the remote serializer, but with a generic message.
Run the shared admission check there so the error names the required
extension IDs. Also state exactly what preflight previews.

Related to nicobailon#2639

* fix: record async runs in run-history.jsonl (parity with foreground executor) (nicobailon#2620)

* fix: record async runs in run-history.jsonl (parity with foreground executor)

recordRun() was called only by the foreground executor, so every async
launch was invisible to the per-agent run census. With asyncByDefault
that is ALL subagent tool calls on a default install: run-history.jsonl
goes permanently stale (verified on a live machine: last entry 09-23,
140 async launches on 10-01 with zero rows).

Record once at the runner's terminal publication (next to the
subagent.run.completed lifecycle event), mirroring foreground semantics:

- agent: agentName (single agent, or parallel:/chain: composite)
- task: single-step task text via backgroundRunHistoryTask() — feeds
  taskHash, redacted on disk; multi-step runs hash the mode label only,
  never per-step prompts
- exitCode: 0 iff state === "complete" (partial => failed, like the
  result payload's own success field)
- terminal flags: stopped/interrupted/timedOut/first processSignal, so
  paused runs record as "interrupted" like an interrupted foreground run;
  a later resume that settles records again — one entry per attempt

The helper is unit-tested; the wiring itself sits in the 5k-line runner
finalize path which has no existing unit seam, verified end-to-end on a
live machine instead.

* address review: per-step history rows, changelog credit, plan-level tests

- P1: multi-step runs now record one row per child step (agent, own
  durationMs, own exitCode, own processSignal) so loadRunsForAgent(agent)
  sees every child; the composite-only row stayed invisible to per-agent
  lookups. Single-step runs keep the exact foreground shape. Mapping
  extracted into planBackgroundRunHistory(), a pure function.
- P2: planBackgroundRunHistory unit-tested (single/multi/duplicate agents,
  run-level flag propagation, per-step signal, agent-name fallback);
  target file 30/30.
- P0: CHANGELOG Unreleased entry with contributor credit (nicobailon#2620).

* address review round 2: skip unrun steps, keep finished children's outcomes

- P1 (unrun children count as failures): steps still 'pending' when a run
  ends (e.g. a chain stopped after a failure) get no row — unrun agents
  accumulate no attempts or failures.
- P1 (completed children get wrong outcomes): run-level terminal flags
  (stopped/interrupted/timedOut) now apply only to steps that did not
  reach terminal success; a child that finished before the run was
  interrupted keeps its 'completed' outcome (recordRun lets those flags
  override a successful exit code).
- Both edges unit-tested; pi-coding-agent-dir.test.ts 32/32.

* address review round 3: self-terminal steps keep their own outcome

A step that already reached its own terminal state (completed, failed,
rejected) keeps its own flags instead of inheriting the run-wide
stopped/interrupted/timedOut — otherwise an earlier failure gets
relabeled when a sibling is later interrupted (recordRun() lets those
flags override the exit code). Self-terminal steps now carry their own
timedOut/stopped flags, so a step that timed out itself still records
as timed_out. Run-wide flags apply only to non-self-terminal steps
(running/paused/partial/stopped).

Unit tests added: earlier-failure-not-relabeled, own-timeout-flag;
pi-coding-agent-dir.test.ts 34/34.

* address review round 4: mirror run timeout onto worktree-setup-failed steps

markParallelGroupSetupFailure already set timedOut on the StepResult but
not on the status step, so run-history recorded those agents as plain
'failed' instead of 'timed_out' when a worktree setup timed out. The
step-level flag is the source of truth the history mapper reads.

* record history rows only for children that actually launched

Status alone cannot decide whether a child ran: stopRunner() relabels
pending steps to stopped, timeoutRunner() to failed+timedOut, and
fail-fast and usage-budget skips to failed — all without ever creating
a child session (the fail-fast skip even sets startedAt). The runner
now threads an explicit launched fact per flat index, recorded at the
three dispatch points, and planBackgroundRunHistory() drops
launched:false steps whatever status they were later relabeled to.

Adds runner-level integration tests (runSubagent + fake child sessions
+ temporary PI_CODING_AGENT_DIR, reading run-history.jsonl back):
stop queued before a two-step chain starts records zero rows; a
fail-fast parallel run records only the child that ran, not its
skipped sibling; a normally completing chain still records one row per
launched step. Also privatizes backgroundRunHistoryTask per review —
its cases are covered through planBackgroundRunHistory rows.

---------

Co-authored-by: jiaminli <[email protected]>

* fix(settings): save builtin agent overrides atomically (nicobailon#2627)

* fix(settings): save builtin agent overrides atomically

* docs: credit contributor for nicobailon#2627

* fix: launch npm-installed external CLIs on Windows without a shell (nicobailon#2647)

* fix: launch npm-installed external CLIs on Windows without a shell

npm installs CLIs such as claude, codex, and cursor-agent as .cmd shims.
Since CVE-2024-27980 Node refuses to spawn a .cmd file with shell: false,
so the external-CLI preflight probes and the run spawn failed with EINVAL.

On Windows, resolve the command and, when it is a .cmd that matches npm's
cmd-shim template for a Node bin, run Node (the shim's bundled node.exe
or the resolved Node runtime) on the shim's script with the original
arguments. A command already ending in a PATHEXT extension is looked up
as written, and a batch file found on PATH keeps that path so its target
resolves from the shim's own directory. Every other .cmd or .bat wrapper
is rejected before it runs; no shell is ever used. Native .exe, POSIX,
and status paths are unchanged.

Fixes nicobailon#2631

* test: use the on-disk shim extension case in the Path lookup assertion

* Test clean installs against Pi 1.0.0 (nicobailon#2648)

* test: cover Pi 1.0 in clean-install smoke

* ci: test clean installs against Pi 1.0

* docs: credit nicobailon#2648 in the changelog

* feat: opt-in command supervision for native Pi children (nicobailon#2598)

* fix: supervise individual child bash commands

* fix: align command smoke fixtures and custom bash activation

* fix: preserve explicitly required command tool with custom bash

* test: publish service PIDs atomically before cleanup checks

* fix: opt in to native command control and preserve custom tools

* test: await live service abort and Windows fixture cleanup

* fix: retain requested custom-backend command availability

* fix: keep command id metadata within schema token budget

* perf: keep command supervision off status paths

Ordinary targeted status no longer reads command state. It added a
synchronous commands.json read per child, plus a status.json reread for
async runs, on every `{ action: "status", id }`, even for children that
never opted into command supervision. Run state does not record that
opt-in, so command inspection stays with the explicit command.status
action, which the docs already point to.

A read-only command status no longer rewrites commands.json. operate()
persisted on every call with a toolCallId, so the child tool's status
check before and after its wait, and the parent's live command.status,
did full atomic writes when nothing changed. It now persists only when
a yield or cancel changes the command's state.

Also share the header, run, signal, and status lines between the
control notice branches (rendered text unchanged) and drop a delivery
test that repeated the adjacent test and the formatter tests.

* perf: keep the subagent schema within its size budget after merging main

Merging main left SubagentParams at 13,080 serialized chars, over the 13,010 budget. Drop the length bounds on toolCallId (the child-side subagent_command tool takes an unbounded string and the id is only a lookup key) and the gate description's restatement of the object fields its own schema already declares. 13,009 chars.

* docs(schema): say only output json gates produce structuredOutput

* test: keep the gate description wording the schema test asserts

---------

Co-authored-by: Nico Bailon <[email protected]>

* chore: tighten unreleased code, tests, and changelog (nicobailon#2649)

- Production: inline single-use helpers (model verification error,
  requestString), simplify internalTools, drop an unreachable settings
  type guard, replace status casts with typeof guards, derive the
  background run-history terminal type from recordRun, trim comments,
  and make childCommandStatePath module-private.
- Tests: remove a duplicate adapter-launch test and a duplicate
  missing/corrupt mission pointer case, drop parsed-JSON duplicates of
  raw-bytes assertions, a fixture self-check, a no-op replace, and
  trailing assertion comments, and replace any/never casts with real
  types where they compile.
- CHANGELOG: shorten the claude-code model/thinking, clean-install, and
  async run-history entries.

* chore(release): v0.75.0

* fix: resolve codex marketplace skills for subagents

* fix: require explicit skill bindings only

---------

Co-authored-by: Nico Bailon <[email protected]>
Co-authored-by: limin <[email protected]>
Co-authored-by: jiaminli <[email protected]>
Co-authored-by: quifox <[email protected]>
Co-authored-by: 周凯 <[email protected]>
Co-authored-by: Jiuai233 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants