feat(core): migrate tool overflow storage from disk to SQLite#1782
Merged
feat(core): migrate tool overflow storage from disk to SQLite#1782
Conversation
Replaces filesystem-based overflow (UUID files in ~/.zeph/overflow/) with a `tool_overflow` SQLite table, eliminating absolute path leakage into LLM context (SEC-JIT-03) and making the database self-contained. Changes: - Add migration 030_tool_overflow.sql with ON DELETE CASCADE FK to conversations - Add SqliteStore methods: save_overflow, load_overflow (conversation-scoped), cleanup_overflow, overflow_size - Replace OverflowConfig.dir with retention_days (default 30) and max_overflow_bytes cap (default 10 MiB, 0 = unlimited) - Update reference format from absolute path to opaque overflow:<uuid> - Add read_overflow native tool for LLM retrieval; exempt from re-overflow - Spawn age-based cleanup task in runner, daemon, and acp entry points - Remove crates/zeph-tools/src/overflow.rs (filesystem backend deleted) - Update debug_dump rendering and docs Fixes #1774
github-actions
bot
added
enhancement
bug-ops
added a commit
that referenced
this pull request
Mar 15, 2026
… migration
OVERFLOW_NOTICE_PREFIX was pointing to the old file-path format
('[full output saved to') while the actual overflow notice format
changed in PR #1782 to '[full output stored as overflow:{uuid}'.
As a result, extract_overflow_ref() always returned None and the JIT
reference was never injected into pruned tool outputs.
- Update OVERFLOW_NOTICE_PREFIX to match current SQLite overflow format
- Update extract_overflow_ref() doc comment (UUID instead of path)
- Update all five JIT reference sites to emit
'[tool output pruned; use read_overflow {uuid} to retrieve]'
- Update tests in summarization.rs and context/mod.rs to use new format
Closes #1818
4 tasks
bug-ops
added a commit
that referenced
this pull request
Mar 15, 2026
… migration (#1826) OVERFLOW_NOTICE_PREFIX was pointing to the old file-path format ('[full output saved to') while the actual overflow notice format changed in PR #1782 to '[full output stored as overflow:{uuid}'. As a result, extract_overflow_ref() always returned None and the JIT reference was never injected into pruned tool outputs. - Update OVERFLOW_NOTICE_PREFIX to match current SQLite overflow format - Update extract_overflow_ref() doc comment (UUID instead of path) - Update all five JIT reference sites to emit '[tool output pruned; use read_overflow {uuid} to retrieve]' - Update tests in summarization.rs and context/mod.rs to use new format Closes #1818
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1774
Summary
Replaces filesystem-based overflow storage (UUID files in
~/.zeph/overflow/) with atool_overflowSQLite table. This eliminates absolute path leakage into LLM context (SEC-JIT-03) and makes the database fully self-contained.tool_overflowSQLite table (migration 031) withON DELETE CASCADEFK to conversationsmaybe_summarize_tool_outputwrites overflow BLOBs to SQLite instead of diskoverflow:<uuid>format — no paths in LLM contextread_overflownative tool allows LLM to retrieve full content by UUID; exempted from re-overflow to prevent infinite retrieval loopsload_overflow(UUID + conversation_id) prevents cross-conversation accessmax_overflow_bytescap (default 10 MiB, 0 = unlimited) prevents unbounded BLOB growthcleanup_overflowon startup in runner, daemon, and acp entry pointscrates/zeph-tools/src/overflow.rsandOverflowConfig.dirfully removedTest plan
save_overflow/load_overflowroundtripextract_overflow_refparsing withoverflow:<uuid>formatread_overflownative tool behavior (found, not found, conversation scoping)cleanup_overflowcorrectnessread_overflowoutput exempt from re-overflow pipelineconversation_idisNonemax_overflow_bytescap enforced before insertcargo +nightly fmt --check— cleancargo clippy --workspace --features full -- -D warnings— cleancargo nextest run --config-file .github/nextest.toml --workspace --features full --lib --bins— 5585 passed