Repository navigation
feat(compression): describe what CCR compression dropped, not just how much - #3635
Conversation
PR governanceThis PR follows the template and is marked ready for human review. |
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
JerrettDavis
left a comment
There was a problem hiding this comment.
Verified the refreshed head against current main: mergeable and conflict-free. The CCR descriptor remains bounded and deterministic, exposes counts rather than paths, and its marker remains retrievable through the generic 24-hex fallback; the full current CI/security matrix is green. Ready for maintainer merge review.
| continue; | ||
| } | ||
| if let Some(label) = extract_error_label(&line.content) { | ||
| error_types.insert(label); |
There was a problem hiding this comment.
The description says error types already visible in kept lines are left out of the descriptor, but this only filters by line number. A KeyError in both kept and dropped lines would still appear in the marker. Should we exclude the kept types too, or is the marker meant to describe all dropped content? A test covering that overlap would help.
There was a problem hiding this comment.
Good catch, thanks. You're right — kept was only used to skip lines by line_number, so it never checked whether the same error type/file was also extractable from a different line that survived. A KeyError in both a kept and a dropped line would indeed show up in the descriptor, which contradicts what I wrote it was doing.
Fixed in e7c81b97: I now build the set of labels/files extractable from the kept lines first, and exclude those from what gets collected off the dropped lines. Also added a direct unit test for summarize_omitted (summarize_omitted_excludes_error_type_and_file_visible_in_kept_lines) covering exactly this overlap — a value present in both a kept and a dropped line is now omitted from the descriptor, while a value present only in dropped lines still shows up.
…w much Extends the CCR marker (`[N lines compressed to M. Retrieve more: hash=...]`) with a short descriptor of the dropped content's error types and source files when extractable, e.g. `[135 lines compressed to 50: 1 file, 3 exception types (E0308, E0425, E0599). Retrieve more: hash=...]`. Additive: an empty descriptor reproduces the old marker byte-for-byte. Three small functions in log_compressor.rs do the work: extract_error_label pulls a rustc code or Python exception name from a line that looks like an error; extract_source_file does the same for file paths (Python `File "..."` lines, rustc `--> file:line:col` lines); summarize_omitted runs both over every dropped line and builds the descriptor, capped at 5 names before falling back to `...`. Only what's absent from the surviving text is described, since the descriptor's job is to justify retrieval, not repeat what's already visible. A blinded 744-run evaluation (124 log/question pairs x old/new marker x Haiku/Sonnet/Opus) found the descriptive marker raises mean correctness on whether a fact is confidently inferable without retrieving: Haiku 0.690 -> 0.952, Sonnet 0.786 -> 0.980, Opus 0.911 -> 1.000. The gain is largest for the weakest model, which is also the one agent harnesses default to most for cost reasons. Full methodology and results are in the PR description. cargo test -p headroom-core log_compressor: 26/26 passing (24 existing + 2 new). cargo fmt and cargo clippy -p headroom-core -- -D warnings clean. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_013ercnWCgmRuB5cZBZripd6
…text
summarize_omitted deduped dropped-line error types/files by line_number
only, not by the extracted value itself. If the same error type or file
appeared in both a kept line and a dropped line, the descriptor still
listed it, contradicting the documented intent ("what retrieval would
add, not what's already visible") and the PR description.
Fix: build the set of labels/files extractable from kept lines first,
then exclude those from the dropped-line pass. Adds a direct unit test
for summarize_omitted covering the overlap case, per review feedback.
cargo test -p headroom-core log_compressor: 27/27 (26 previous + 1 new).
cargo fmt and cargo clippy -p headroom-core -- -D warnings clean.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_013ercnWCgmRuB5cZBZripd6
49f4f1d to
b1fd0b8
Compare
JerrettDavis
left a comment
There was a problem hiding this comment.
Re-reviewed exact head b1fd0b8 after the clean rebase. The cumulative contributor patch is unchanged, focused log-compressor tests, formatting, and clippy pass locally, and all required checks are green.
🤖 I have created a release *beep* *boop* --- ## [0.39.0](v0.38.0...v0.39.0) (2026-09-25) ### Features * **compression:** describe what CCR compression dropped, not just how much ([#3635](#3635)) ([62d1cc0](62d1cc0)) * **hooks:** add protect_messages, a hard per-message compression veto ([#3772](#3772)) ([d17ac6a](d17ac6a)) * **opencode:** exclude hosts from transport plugin routing ([#3668](#3668)) ([0024b57](0024b57)) * **proxy:** let /transformations/feed omit message bodies for number-only pollers ([#3672](#3672)) ([46ac52d](46ac52d)) * **savings:** carry exact cache-read cost in history rollups ([#3734](#3734)) ([1cb779e](1cb779e)) * **savings:** show the new-input savings basis beside the whole-wire figure ([#3485](#3485)) ([e92cccc](e92cccc)) * **savings:** track lifetime output spend so a bill-share rate has a denominator ([#3520](#3520)) ([c766bdb](c766bdb)) * **sdk:** support per-call config override in compress() ([#3370](#3370)) ([c3e8a1e](c3e8a1e)) * **wrap:** support 1m context in vscode claude ([#3361](#3361)) ([fc5a09e](fc5a09e)) ### Bug Fixes * **anthropic:** reject ping-only SSE from buffered CCR ([#3682](#3682)) ([000fefc](000fefc)) * **anthropic:** stop the spurious compaction warning and drop dead handler code ([#3678](#3678)) ([6e71267](6e71267)) * **backends/litellm:** stop dropping image blocks from /v1/messages requests ([#3753](#3753)) ([577336d](577336d)) * **binaries:** fail closed when a download carries no sha256 pin ([#3724](#3724)) ([26a2c49](26a2c49)) * **cache/google:** tolerate timezone-aware cache expiry timestamps ([#3210](#3210)) ([7ce2580](7ce2580)) * **cache:** default CCR payload previews off; harden the log when they are on ([#3728](#3728)) ([5cb87bc](5cb87bc)) * **cache:** never batch-compress already-forwarded small Responses outputs in cache mode ([#3756](#3756)) ([ab62b9e](ab62b9e)) * **cache:** preserve cache_control schema properties ([#3348](#3348)) ([b84c4c9](b84c4c9)) * **cache:** preserve the forwarded OpenAI chat prefix across turns ([#3757](#3757)) ([deca575](deca575)) * **cache:** stop prefix transforms from busting the provider cache ([#3735](#3735)) ([a2bf5ed](a2bf5ed)) * **cache:** stop skipping tool compaction on a pinned tools array ([#3750](#3750)) ([a6a9cef](a6a9cef)) * **cli:** keep non-ASCII working directory names out of the init profile slug ([#3718](#3718)) ([f3d95ee](f3d95ee)) * **cli:** report anonymous beacon status accurately in the banner, log, and /stats ([#3533](#3533)) ([fa9edb3](fa9edb3)) * **content_detector:** detect CMTrace (SCCM/Intune) logs as BUILD_OUTPUT ([#3712](#3712)) ([6feb1fb](6feb1fb)) * **dashboard:** separate rolling cache economics by owner ([#3319](#3319)) ([a29162b](a29162b)), closes [#960](#960) * **docker:** support OpenCode in native wrapper ([#3547](#3547)) ([d90dadf](d90dadf)) * fall back for non-admin Windows task installs ([#3437](#3437)) ([b958509](b958509)) * **graph:** fetch and extract the zip codebase-memory-mcp asset on Windows ([#3730](#3730)) ([b776983](b776983)) * **grok:** preserve xAI model context metadata ([#3312](#3312)) ([#3315](#3315)) ([c81378c](c81378c)) * **init:** raise the ensure hook's timeout above cold-start latency ([#3438](#3438)) ([9a11109](9a11109)) * **install:** apply managed env vars added after a deployment was installed ([#3740](#3740)) ([a4cb2bc](a4cb2bc)) * **install:** wait for the old process to stop before restarting ([#3670](#3670)) ([11c7320](11c7320)) * **kompress:** bound the inference deadline to the request, not each block ([#3693](#3693)) ([7f2766c](7f2766c)) * **kompress:** keep the boolean connectives that hold a condition together ([#3687](#3687)) ([bf290ba](bf290ba)) * **learn:** treat an unreadable project memory dir as absent instead of crashing ([#3710](#3710)) ([ff60d57](ff60d57)) * **mcp:** accept an empty install ledger instead of failing mutations as malformed ([#3235](#3235)) ([19ddfe1](19ddfe1)) * **memory/graph:** apply relation_type filter to both sides of a BOTH subgraph query ([#3236](#3236)) ([1a6f941](1a6f941)) * **memory:** fail closed for unresolved tools ([#3655](#3655)) ([7c3cbc8](7c3cbc8)) * **memory:** make extract_system_prompt concatenative, not first-match ([#3644](#3644)) ([3f3cf19](3f3cf19)) * **memory:** tolerate None message content in inline memory parser ([#3211](#3211)) ([5ccec67](5ccec67)) * **metrics:** split rate-limited by source and label failures by provider ([#3707](#3707)) ([5ff4ea1](5ff4ea1)) * **openclaw:** declare transcriptSemantics and implement commitTurn ([#3442](#3442)) ([a28dd4e](a28dd4e)) * **perf:** make budget checks constant-time ([#3368](#3368)) ([ed08069](ed08069)) * **perf:** make rate limiter bucket checks constant-time ([#3373](#3373)) ([a8bd9be](a8bd9be)) * preserve null and empty strings in lossless CSV compaction ([#3665](#3665)) ([7ad9f80](7ad9f80)) * **proxy:** admit container host gateway on compress routes ([#3715](#3715)) ([a8ae0a6](a8ae0a6)) * **proxy:** bound request-log message payloads to the feed window; trim glibc too ([#3717](#3717)) ([994a89f](994a89f)) * **proxy:** count a ceiling-stopped stream exactly instead of estimating its text ([#3771](#3771)) ([b36e59a](b36e59a)) * **proxy:** discard partial SSE tails during finalization ([#3411](#3411)) ([6c0e817](6c0e817)) * **proxy:** enforce configured TPM limits ([#3350](#3350)) ([138736c](138736c)) * **proxy:** fall back when forced Kompress is cold ([#3381](#3381)) ([ca54c2a](ca54c2a)) * **proxy:** make the output shaper actually reduce output in the default mode ([#3773](#3773)) ([c029298](c029298)) * **proxy:** partition response cache by upstream ([#3349](#3349)) ([dfdc725](dfdc725)) * **proxy:** pin the validated address so DNS rebinding cannot move an upstream ([#3731](#3731)) ([a102e05](a102e05)) * **proxy:** preserve Claude Code auto-mode protocol ([#3784](#3784)) ([12c1579](12c1579)) * **proxy:** preserve empty transport error context ([#3675](#3675)) ([ca02f28](ca02f28)) * **proxy:** preserve OpenCode Headroom retrieve results ([#3782](#3782)) ([c7ebaed](c7ebaed)) * **proxy:** protect Codex exec reads whose argument is a JavaScript object literal ([#3737](#3737)) ([ecb5e5a](ecb5e5a)) * **proxy:** reject Anthropic batch operations on Copilot ([#3318](#3318)) ([871bbde](871bbde)) * **proxy:** repair client-side tool_search tool_reference blocks ([#3443](#3443)) ([f12e3fb](f12e3fb)) * **proxy:** resolve Codex custom_tool_call names for Responses tool exclusion ([#3755](#3755)) ([55c78de](55c78de)) * **proxy:** restrict urlopen SSL context ALPN to http/1.1 ([#3567](#3567)) ([5baa439](5baa439)) * **proxy:** scope OpenAI rate limits by api-key ([#3365](#3365)) ([b9e8462](b9e8462)) * **proxy:** stamp x-headroom-* compression metrics on streaming responses ([#3769](#3769)) ([2c4dc44](2c4dc44)) * remove deprecated Apache license classifier ([#3566](#3566)) ([b2d9275](b2d9275)) * **savings:** report negative savings instead of flooring them at zero ([#3744](#3744)) ([1455f00](1455f00)) * **sdk:** enforce dormant public type tests ([#3424](#3424)) ([95cbbb8](95cbbb8)) * **sdk:** preserve Anthropic image blocks in message conversion instead of dropping the turn ([#3439](#3439)) ([63676e6](63676e6)) * **sdk:** preserve proxy authentication alongside provider credentials ([#3450](#3450)) ([ca4b47a](ca4b47a)) * **security:** create credential files private instead of narrowing after write ([#3215](#3215)) ([9b8cae8](9b8cae8)) * **security:** reject unauthenticated public proxy binds ([#3316](#3316)) ([b0c19a2](b0c19a2)) * **smart-crusher:** disclose dropped scalar-array items with a retrievable ccr sentinel ([#3692](#3692)) ([c1fc84a](c1fc84a)) * **spreadsheet:** stop .xls integer rendering fabricating digits above 2^53 ([#3704](#3704)) ([e8a5109](e8a5109)) * **spreadsheet:** stop inventing digits for .xls cells above 2**53 ([#3700](#3700)) ([1a7c5bb](1a7c5bb)) * **tests:** ask litellm for a model it prices instead of naming a retired one ([#3722](#3722)) ([0ad5e68](0ad5e68)) * **tool-search:** detect every wire shape of client-side deferral ([#3746](#3746)) ([665b73d](665b73d)) * **tool-search:** don't orphan deferred tools, and stop overstating the saving ([#3739](#3739)) ([85f9e01](85f9e01)) * **tool-search:** make the core-tools override authoritative and the warning recurrent ([#3745](#3745)) ([a9757c9](a9757c9)) * **transforms:** detect fixed-width command output as tabular and keep it out of Kompress ([#3786](#3786)) ([6c9aef1](6c9aef1)) * **transforms:** stop embedded JSON pre-pass short-circuiting HTML extraction ([#3620](#3620)) ([195910b](195910b)) * **transforms:** stop routing timestamped log rows as grep matches ([#3748](#3748)) ([d971f7c](d971f7c)) * **wrap:** handle SIGHUP on the shared proxy paths, not just `claude` ([#3331](#3331)) ([c4df2dd](c4df2dd)) * **wrap:** stop blocking the launch on the Serena pre-index ([#3557](#3557)) ([df8ebdf](df8ebdf)) ### Performance Improvements * **cache:** drop eager full-backend TTL sweep on every new-key store() ([#3256](#3256)) ([2fed685](2fed685)) * **cache:** LRU-bound the compression-feedback tool-pattern map ([#3342](#3342)) ([2d10b10](2d10b10)) * **memory:** reindex FTS5 a page at a time instead of a record at a time ([#3466](#3466)) ([3e36aef](3e36aef)) * **providers:** cache Google/Cohere token counters across requests ([#3273](#3273)) ([cc6a07c](cc6a07c)) * **relevance:** order BM25 query terms once, not once per document ([#3468](#3468)) ([6880984](6880984)) ### Dependencies * bump dotenv from 17.4.2 to 18.0.1 in /sdk/typescript ([#3765](#3765)) ([56edfbc](56edfbc)) * bump ruff from 0.16.7 to 0.16.8 in the pip-minor-patch group ([#3761](#3761)) ([b9ded1a](b9ded1a)) * bump the cargo-minor-patch group with 3 updates ([#3762](#3762)) ([fbbe8b8](fbbe8b8)) * bump the npm-minor-patch group across 2 directories with 19 updates ([#3763](#3763)) ([54357ff](54357ff)) * bump the npm-minor-patch group across 3 directories with 9 updates ([#3764](#3764)) ([6311332](6311332)) ### Code Refactoring * **sdk:** route _fetch through rawFetch instead of duplicating it ([#3473](#3473)) ([b2b5847](b2b5847)) * **smart-crusher:** drop the dead crush_object clone from the object path ([#3705](#3705)) ([7221724](7221724)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Description
When the log compressor drops lines, it leaves behind a CCR marker:
[N lines compressed to M. Retrieve more: hash=...]. Today that marker says how much was cut but nothing about what, so an agent deciding whether to callheadroom_retrieveis guessing. This PR adds a short descriptor of the dropped content's error types and source files, when they're extractable:An agent chasing a
KeyErrorthat sees the marker only mentions Rust compiler codes now has a real reason to skip retrieval, and vice versa when its target error is named.I don't have a linked issue for this — I couldn't find a prior discussion of the marker's information content, so I'm opening this as a concrete proposal with data behind it rather than a spec-first change. Happy to split this into an issue first if that's preferred, or to adjust the approach based on maintainer feedback.
Type of Change
Changes Made
extract_error_labeltocrates/headroom-core/src/transforms/log_compressor.rs: pulls a rustc code (E0308) or Python exception name (KeyError) out of a line that looks like an error.extract_source_file: same idea for file paths (PythonFile "..."lines, rustc--> file:line:col).summarize_omitted: runs both extractors over every line about to be dropped and builds the descriptor appended to the marker, capped at 5 names shown before falling back to....Testing
cargo test -p headroom-core log_compressor)cargo clippy -p headroom-core --all-targets -- -D warnings)Test Output
Real Behavior Proof
main.cargo test -p headroom-core log_compressor;cargo fmt -p headroom-core -- --check;cargo clippy -p headroom-core --all-targets -- -D warnings; also ran the compressor against a real capturedcargo buildfailure log to confirm the new marker text renders correctly outside the unit tests.Evaluation (design justification, not a repo test)
62 real Python test-failure logs (genuine bugs, actually run, not hand-crafted) produced 124 log/question pairs — for each log: how many distinct files, and how many distinct error types, can you be certain are present? Each pair was shown twice, once per marker version, to Haiku, Sonnet, and Opus independently — 744 blinded runs, no memory across calls. A deterministic rubric scored answers: 0 for wrong/overclaimed, 0.5 for an unnecessary hedge, 1 for a defensible correct answer.
Every model improves; the gain is largest for the weakest model. Haiku hedged on 77/124 questions with the old marker, dropping to 12/124 with the new one — this matters because agent harnesses increasingly default to cheaper, smaller models to control cost and latency, so the marker hands the most help to the model that needs it most.
Limits: one log format at this scale (others the log compressor handles weren't checked here); all three judges are Claude-family models; measures one retrieval decision, not full task success. Full harness, corpus, and per-call transcripts are available on request — left out of this diff to keep it focused on the code change.
Runtime Rollout Safety
Review Readiness
Checklist
CHANGELOG.md