test: guard prompt-prefix byte stability at boot and desktop rebind level#6094
Conversation
…evel Final defense line for the provider prefix cache, closing the CI gaps the desktop-cost triage identified: nothing verified that (a) prompt composition is deterministic and (b) the desktop rebind path preserves request bytes. - TestBuildComposesByteStableSystemPrompt: two Builds over the same workspace/config must compose byte-identical system prompts; a failure prints a window around the first diverging byte so the drifting section is named. Probes are covered cross-process by the persisted-snapshot tests; this pins the rest of the composition (memory, skills index, output style, policies). - TestRebindKeepsRequestPrefixBytes: after the desktop rebind shape (LoadSession + sessionWithFreshSystemPrompt + Resume on a fresh controller), the next request's leading messages must be byte-identical to the request sent before the rebind. - TestRebindWithDriftedPromptBreaksRequestPrefix pins the failure mode the guard exists for (a drifted prompt breaks the prefix), so a future swap-policy change updates the guards consciously.
github-actions
Bot
added
v2
Review feedback on the rebind guard: comparing after[:len(before)] against the FIRST turn's request only proved the system prompt and first user message stayed stable — the resumed prefix's assistant/tool messages were never compared. Strengthen to the full comparison: from one transcript saved after turn one, run the same follow-up turn twice — once on the original controller (no rebind, the cache-warm baseline) and once after the desktop rebind path — and require the two requests to be byte-identical END TO END. The rebind now loads an independent copy of the saved transcript, since the baseline turn autosaves onto the original path. The drifted-prompt canary gets the same baseline structure and additionally pins the divergence to the leading system message.
|
Both findings addressed. P1 (cache-impact red): fixed before this comment — the PR body's P2 (guard weaker than its claim): correct, and fixed in 1192fad. The old comparison ( Worth noting: the strengthened comparison immediately caught a bug — in the test itself: the baseline turn autosaves onto the shared session path, so the rebind was loading a transcript that already contained the baseline exchange. The rebind now loads an independent copy of the after-turn-one save ( Verification: full desktop suite green (62s) including both rewritten tests; |
1 participant
Summary
治本工程收尾：给 provider 前缀缓存加最终防线测试，补上 desktop 成本排查时确认的两个 CI 盲区——此前没有任何测试验证 (a) prompt 组装是确定性的、(b) desktop rebind 路径保持请求字节不变。
TestBuildComposesByteStableSystemPrompt(internal/boot): twoBuilds over the same workspace/config must compose byte-identical system prompts. On failure it prints a window around the first diverging byte, so the drifting prompt section is named directly instead of dumping both prompts. Probe stability across processes is already covered by the persisted-snapshot tests from fix(environment): persist probe snapshots so the prompt prefix stays byte-stable #6093; this pins the rest of the composition (memory compose, skills index, output style, workspace line, policies) against future nondeterminism.TestRebindKeepsRequestPrefixBytes(desktop): drives the real desktop rebind shape —agent.LoadSession+sessionWithFreshSystemPrompt+Resumeon a freshly built controller — with a request-capturing provider, and asserts the next turn's leading request messages are byte-identical to the request sent before the rebind. This is the prefix-cache invariant itself: if it holds, a desktop rebuild costs zero cache misses.TestRebindWithDriftedPromptBreaksRequestPrefixpins the failure mode the guard exists for: a drifted fresh prompt rewrites the first message and the prefix diverges. If the swap policy ever changes to keep the persisted prompt for resumed conversations, this test flips and forces a conscious update of both guards.Tests only — no production code changes.
Verification
go test ./internal/boot(16s) and full desktop suite (39s) green;go vetclean.Cache impact
Cache-impact: none - test-only change; adds the guards that future cache-sensitive changes will be caught by.
Cache-guard: this PR IS the guard — boot-level composition determinism plus desktop rebind request-prefix byte equality.
System-prompt-review: requested from @SivanCola — test-only change (internal/boot/* glob matches the new test file); no provider-visible content changes, the new test pins existing composition bytes.
Refs #2945, #5614; builds on #6093.