Skip to content

feat(compression): describe what CCR compression dropped, not just how much - #3635

Merged
JerrettDavis merged 2 commits into
headroomlabs-ai:mainfrom
SANGITA-D:feat/log-compressor-error-descriptor
Sep 24, 2026
Merged

JerrettDavis merged 2 commits into
headroomlabs-ai:mainfrom
SANGITA-D:feat/log-compressor-error-descriptor

Conversation

@SANGITA-D

@SANGITA-D SANGITA-D commented Sep 17, 2026 •

Copy link
Copy Markdown

Description

When the log compressor drops lines, it leaves behind a CCR marker: [N lines compressed to M. Retrieve more: hash=...]. Today that marker says how much was cut but nothing about what, so an agent deciding whether to call headroom_retrieve is guessing. This PR adds a short descriptor of the dropped content's error types and source files, when they're extractable:

Before: [135 lines compressed to 50. Retrieve more: hash=abc123]
After:  [135 lines compressed to 50: 1 file, 3 exception types (E0308, E0425, E0599). Retrieve more: hash=abc123]

An agent chasing a KeyError that sees the marker only mentions Rust compiler codes now has a real reason to skip retrieval, and vice versa when its target error is named.

I don't have a linked issue for this — I couldn't find a prior discussion of the marker's information content, so I'm opening this as a concrete proposal with data behind it rather than a spec-first change. Happy to split this into an issue first if that's preferred, or to adjust the approach based on maintainer feedback.

Type of Change

  • New feature (non-breaking change that adds functionality)

Changes Made

  • Add extract_error_label to crates/headroom-core/src/transforms/log_compressor.rs: pulls a rustc code (E0308) or Python exception name (KeyError) out of a line that looks like an error.
  • Add extract_source_file: same idea for file paths (Python File "..." lines, rustc --> file:line:col).
  • Add summarize_omitted: runs both extractors over every line about to be dropped and builds the descriptor appended to the marker, capped at 5 names shown before falling back to ....
  • Descriptor only covers what's absent from the surviving text — an error type that also appears in a kept line isn't repeated, since the descriptor's job is to justify retrieval, not restate what's already visible.
  • Additive and backward-compatible: if nothing is extractable, the descriptor is empty and the marker is byte-identical to today's output.
  • 2 new unit tests covering descriptor presence and the empty-descriptor fallback.

Testing

  • Unit tests pass (cargo test -p headroom-core log_compressor)
  • Linting passes (cargo clippy -p headroom-core --all-targets -- -D warnings)
  • New tests added for new functionality
  • Manual testing performed (real captured cargo failure, not just unit fixtures)

Test Output

$ cargo test -p headroom-core log_compressor
running 26 tests
test transforms::log_compressor::tests::ccr_marker_emitted_when_thresholds_clear ... ok
test transforms::log_compressor::tests::ccr_marker_descriptor_empty_when_nothing_extractable ... ok
test transforms::log_compressor::tests::ccr_marker_includes_error_types_and_files_when_available ... ok
... (23 more, all pre-existing)
test result: ok. 26 passed; 0 failed; 0 ignored; 0 measured; 901 filtered out

$ cargo fmt -p headroom-core -- --check   # clean, no diff
$ cargo clippy -p headroom-core --all-targets -- -D warnings   # clean, no warnings

Real Behavior Proof

  • Environment: macOS, local Rust workspace build, cargo test/fmt/clippy against current main.
  • Exact command / steps: cargo test -p headroom-core log_compressor; cargo fmt -p headroom-core -- --check; cargo clippy -p headroom-core --all-targets -- -D warnings; also ran the compressor against a real captured cargo build failure log to confirm the new marker text renders correctly outside the unit tests.
  • Observed result: 26/26 tests pass, fmt clean, clippy clean, real-log marker output matches the documented format.
  • Separately, to check whether the descriptor actually changes an agent's behavior (not just that it compiles), I ran a blinded evaluation described below — not part of this repo's test suite, offered as evidence for the design decision.
  • Not tested: log formats other than Python test output and Rust/cargo output at this evaluation's scale; non-Claude judge models; end-to-end multi-turn task success (this measures one isolated retrieval decision).

Evaluation (design justification, not a repo test)

62 real Python test-failure logs (genuine bugs, actually run, not hand-crafted) produced 124 log/question pairs — for each log: how many distinct files, and how many distinct error types, can you be certain are present? Each pair was shown twice, once per marker version, to Haiku, Sonnet, and Opus independently — 744 blinded runs, no memory across calls. A deterministic rubric scored answers: 0 for wrong/overclaimed, 0.5 for an unnecessary hedge, 1 for a defensible correct answer.

model     old marker   new marker   gap
Haiku     0.690        0.952        +0.262
Sonnet    0.786        0.980        +0.194
Opus      0.911        1.000        +0.089

Every model improves; the gain is largest for the weakest model. Haiku hedged on 77/124 questions with the old marker, dropping to 12/124 with the new one — this matters because agent harnesses increasingly default to cheaper, smaller models to control cost and latency, so the marker hands the most help to the model that needs it most.

Limits: one log format at this scale (others the log compressor handles weren't checked here); all three judges are Claude-family models; measures one retrieval decision, not full task success. Full harness, corpus, and per-call transcripts are available on request — left out of this diff to keep it focused on the code change.

Runtime Rollout Safety

  • Rollout-managed feature(s): none — no new flag, always active wherever the CCR marker is already emitted.
  • Minimum rollout channel: n/a.
  • Stable/default behavior changed: marker text gains an optional descriptor clause; unchanged when nothing is extractable (byte-identical to the current marker in that case).
  • Kill switch / disable path: none needed since there's no behavior change when extraction finds nothing; reverting this commit fully removes the feature.
  • Unsafe override required: no.
  • Qualification impact: none — purely additive text in an existing marker, no change to compression decisions, byte counts, or the retrieval mechanism itself.
  • Rollback path: revert this commit.

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (no user-facing docs currently describe the marker's format in detail; happy to add if maintainers point me at the right place)
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I did not edit CHANGELOG.md

@github-actions

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added the status: ready for review Pull request body is complete and the author marked it ready for human review label Sep 17, 2026
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the refreshed head against current main: mergeable and conflict-free. The CCR descriptor remains bounded and deterministic, exposes counts rather than paths, and its marker remains retrievable through the generic 24-hex fallback; the full current CI/security matrix is green. Ready for maintainer merge review.

continue;
}
if let Some(label) = extract_error_label(&line.content) {
error_types.insert(label);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description says error types already visible in kept lines are left out of the descriptor, but this only filters by line number. A KeyError in both kept and dropped lines would still appear in the marker. Should we exclude the kept types too, or is the marker meant to describe all dropped content? A test covering that overlap would help.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. You're right — kept was only used to skip lines by line_number, so it never checked whether the same error type/file was also extractable from a different line that survived. A KeyError in both a kept and a dropped line would indeed show up in the descriptor, which contradicts what I wrote it was doing.

Fixed in e7c81b97: I now build the set of labels/files extractable from the kept lines first, and exclude those from what gets collected off the dropped lines. Also added a direct unit test for summarize_omitted (summarize_omitted_excludes_error_type_and_file_visible_in_kept_lines) covering exactly this overlap — a value present in both a kept and a dropped line is now omitted from the descriptor, while a value present only in dropped lines still shows up.

Sangita Dutta and others added 2 commits September 21, 2026 19:52
…w much

Extends the CCR marker (`[N lines compressed to M. Retrieve more: hash=...]`)
with a short descriptor of the dropped content's error types and source
files when extractable, e.g. `[135 lines compressed to 50: 1 file, 3
exception types (E0308, E0425, E0599). Retrieve more: hash=...]`. Additive:
an empty descriptor reproduces the old marker byte-for-byte.

Three small functions in log_compressor.rs do the work: extract_error_label
pulls a rustc code or Python exception name from a line that looks like an
error; extract_source_file does the same for file paths (Python `File "..."`
lines, rustc `--> file:line:col` lines); summarize_omitted runs both over
every dropped line and builds the descriptor, capped at 5 names before
falling back to `...`. Only what's absent from the surviving text is
described, since the descriptor's job is to justify retrieval, not repeat
what's already visible.

A blinded 744-run evaluation (124 log/question pairs x old/new marker x
Haiku/Sonnet/Opus) found the descriptive marker raises mean correctness on
whether a fact is confidently inferable without retrieving: Haiku 0.690 ->
0.952, Sonnet 0.786 -> 0.980, Opus 0.911 -> 1.000. The gain is largest for
the weakest model, which is also the one agent harnesses default to most
for cost reasons. Full methodology and results are in the PR description.

cargo test -p headroom-core log_compressor: 26/26 passing (24 existing + 2
new). cargo fmt and cargo clippy -p headroom-core -- -D warnings clean.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_013ercnWCgmRuB5cZBZripd6
…text

summarize_omitted deduped dropped-line error types/files by line_number
only, not by the extracted value itself. If the same error type or file
appeared in both a kept line and a dropped line, the descriptor still
listed it, contradicting the documented intent ("what retrieval would
add, not what's already visible") and the PR description.

Fix: build the set of labels/files extractable from kept lines first,
then exclude those from the dropped-line pass. Adds a direct unit test
for summarize_omitted covering the overlap case, per review feedback.

cargo test -p headroom-core log_compressor: 27/27 (26 previous + 1 new).
cargo fmt and cargo clippy -p headroom-core -- -D warnings clean.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_013ercnWCgmRuB5cZBZripd6
@JerrettDavis
JerrettDavis force-pushed the feat/log-compressor-error-descriptor branch from 49f4f1d to b1fd0b8 Compare September 22, 2026 00:54

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed exact head b1fd0b8 after the clean rebase. The cumulative contributor patch is unchanged, focused log-compressor tests, formatting, and clippy pass locally, and all required checks are green.

@JerrettDavis
JerrettDavis merged commit 62d1cc0 into headroomlabs-ai:main Sep 24, 2026
46 checks passed
JerrettDavis added a commit that referenced this pull request Sep 25, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.39.0](v0.38.0...v0.39.0)
(2026-09-25)


### Features

* **compression:** describe what CCR compression dropped, not just how
much ([#3635](#3635))
([62d1cc0](62d1cc0))
* **hooks:** add protect_messages, a hard per-message compression veto
([#3772](#3772))
([d17ac6a](d17ac6a))
* **opencode:** exclude hosts from transport plugin routing
([#3668](#3668))
([0024b57](0024b57))
* **proxy:** let /transformations/feed omit message bodies for
number-only pollers
([#3672](#3672))
([46ac52d](46ac52d))
* **savings:** carry exact cache-read cost in history rollups
([#3734](#3734))
([1cb779e](1cb779e))
* **savings:** show the new-input savings basis beside the whole-wire
figure
([#3485](#3485))
([e92cccc](e92cccc))
* **savings:** track lifetime output spend so a bill-share rate has a
denominator
([#3520](#3520))
([c766bdb](c766bdb))
* **sdk:** support per-call config override in compress()
([#3370](#3370))
([c3e8a1e](c3e8a1e))
* **wrap:** support 1m context in vscode claude
([#3361](#3361))
([fc5a09e](fc5a09e))


### Bug Fixes

* **anthropic:** reject ping-only SSE from buffered CCR
([#3682](#3682))
([000fefc](000fefc))
* **anthropic:** stop the spurious compaction warning and drop dead
handler code
([#3678](#3678))
([6e71267](6e71267))
* **backends/litellm:** stop dropping image blocks from /v1/messages
requests
([#3753](#3753))
([577336d](577336d))
* **binaries:** fail closed when a download carries no sha256 pin
([#3724](#3724))
([26a2c49](26a2c49))
* **cache/google:** tolerate timezone-aware cache expiry timestamps
([#3210](#3210))
([7ce2580](7ce2580))
* **cache:** default CCR payload previews off; harden the log when they
are on
([#3728](#3728))
([5cb87bc](5cb87bc))
* **cache:** never batch-compress already-forwarded small Responses
outputs in cache mode
([#3756](#3756))
([ab62b9e](ab62b9e))
* **cache:** preserve cache_control schema properties
([#3348](#3348))
([b84c4c9](b84c4c9))
* **cache:** preserve the forwarded OpenAI chat prefix across turns
([#3757](#3757))
([deca575](deca575))
* **cache:** stop prefix transforms from busting the provider cache
([#3735](#3735))
([a2bf5ed](a2bf5ed))
* **cache:** stop skipping tool compaction on a pinned tools array
([#3750](#3750))
([a6a9cef](a6a9cef))
* **cli:** keep non-ASCII working directory names out of the init
profile slug
([#3718](#3718))
([f3d95ee](f3d95ee))
* **cli:** report anonymous beacon status accurately in the banner, log,
and /stats
([#3533](#3533))
([fa9edb3](fa9edb3))
* **content_detector:** detect CMTrace (SCCM/Intune) logs as
BUILD_OUTPUT
([#3712](#3712))
([6feb1fb](6feb1fb))
* **dashboard:** separate rolling cache economics by owner
([#3319](#3319))
([a29162b](a29162b)),
closes [#960](#960)
* **docker:** support OpenCode in native wrapper
([#3547](#3547))
([d90dadf](d90dadf))
* fall back for non-admin Windows task installs
([#3437](#3437))
([b958509](b958509))
* **graph:** fetch and extract the zip codebase-memory-mcp asset on
Windows
([#3730](#3730))
([b776983](b776983))
* **grok:** preserve xAI model context metadata
([#3312](#3312))
([#3315](#3315))
([c81378c](c81378c))
* **init:** raise the ensure hook's timeout above cold-start latency
([#3438](#3438))
([9a11109](9a11109))
* **install:** apply managed env vars added after a deployment was
installed
([#3740](#3740))
([a4cb2bc](a4cb2bc))
* **install:** wait for the old process to stop before restarting
([#3670](#3670))
([11c7320](11c7320))
* **kompress:** bound the inference deadline to the request, not each
block ([#3693](#3693))
([7f2766c](7f2766c))
* **kompress:** keep the boolean connectives that hold a condition
together
([#3687](#3687))
([bf290ba](bf290ba))
* **learn:** treat an unreadable project memory dir as absent instead of
crashing
([#3710](#3710))
([ff60d57](ff60d57))
* **mcp:** accept an empty install ledger instead of failing mutations
as malformed
([#3235](#3235))
([19ddfe1](19ddfe1))
* **memory/graph:** apply relation_type filter to both sides of a BOTH
subgraph query
([#3236](#3236))
([1a6f941](1a6f941))
* **memory:** fail closed for unresolved tools
([#3655](#3655))
([7c3cbc8](7c3cbc8))
* **memory:** make extract_system_prompt concatenative, not first-match
([#3644](#3644))
([3f3cf19](3f3cf19))
* **memory:** tolerate None message content in inline memory parser
([#3211](#3211))
([5ccec67](5ccec67))
* **metrics:** split rate-limited by source and label failures by
provider
([#3707](#3707))
([5ff4ea1](5ff4ea1))
* **openclaw:** declare transcriptSemantics and implement commitTurn
([#3442](#3442))
([a28dd4e](a28dd4e))
* **perf:** make budget checks constant-time
([#3368](#3368))
([ed08069](ed08069))
* **perf:** make rate limiter bucket checks constant-time
([#3373](#3373))
([a8bd9be](a8bd9be))
* preserve null and empty strings in lossless CSV compaction
([#3665](#3665))
([7ad9f80](7ad9f80))
* **proxy:** admit container host gateway on compress routes
([#3715](#3715))
([a8ae0a6](a8ae0a6))
* **proxy:** bound request-log message payloads to the feed window; trim
glibc too
([#3717](#3717))
([994a89f](994a89f))
* **proxy:** count a ceiling-stopped stream exactly instead of
estimating its text
([#3771](#3771))
([b36e59a](b36e59a))
* **proxy:** discard partial SSE tails during finalization
([#3411](#3411))
([6c0e817](6c0e817))
* **proxy:** enforce configured TPM limits
([#3350](#3350))
([138736c](138736c))
* **proxy:** fall back when forced Kompress is cold
([#3381](#3381))
([ca54c2a](ca54c2a))
* **proxy:** make the output shaper actually reduce output in the
default mode
([#3773](#3773))
([c029298](c029298))
* **proxy:** partition response cache by upstream
([#3349](#3349))
([dfdc725](dfdc725))
* **proxy:** pin the validated address so DNS rebinding cannot move an
upstream
([#3731](#3731))
([a102e05](a102e05))
* **proxy:** preserve Claude Code auto-mode protocol
([#3784](#3784))
([12c1579](12c1579))
* **proxy:** preserve empty transport error context
([#3675](#3675))
([ca02f28](ca02f28))
* **proxy:** preserve OpenCode Headroom retrieve results
([#3782](#3782))
([c7ebaed](c7ebaed))
* **proxy:** protect Codex exec reads whose argument is a JavaScript
object literal
([#3737](#3737))
([ecb5e5a](ecb5e5a))
* **proxy:** reject Anthropic batch operations on Copilot
([#3318](#3318))
([871bbde](871bbde))
* **proxy:** repair client-side tool_search tool_reference blocks
([#3443](#3443))
([f12e3fb](f12e3fb))
* **proxy:** resolve Codex custom_tool_call names for Responses tool
exclusion
([#3755](#3755))
([55c78de](55c78de))
* **proxy:** restrict urlopen SSL context ALPN to http/1.1
([#3567](#3567))
([5baa439](5baa439))
* **proxy:** scope OpenAI rate limits by api-key
([#3365](#3365))
([b9e8462](b9e8462))
* **proxy:** stamp x-headroom-* compression metrics on streaming
responses
([#3769](#3769))
([2c4dc44](2c4dc44))
* remove deprecated Apache license classifier
([#3566](#3566))
([b2d9275](b2d9275))
* **savings:** report negative savings instead of flooring them at zero
([#3744](#3744))
([1455f00](1455f00))
* **sdk:** enforce dormant public type tests
([#3424](#3424))
([95cbbb8](95cbbb8))
* **sdk:** preserve Anthropic image blocks in message conversion instead
of dropping the turn
([#3439](#3439))
([63676e6](63676e6))
* **sdk:** preserve proxy authentication alongside provider credentials
([#3450](#3450))
([ca4b47a](ca4b47a))
* **security:** create credential files private instead of narrowing
after write
([#3215](#3215))
([9b8cae8](9b8cae8))
* **security:** reject unauthenticated public proxy binds
([#3316](#3316))
([b0c19a2](b0c19a2))
* **smart-crusher:** disclose dropped scalar-array items with a
retrievable ccr sentinel
([#3692](#3692))
([c1fc84a](c1fc84a))
* **spreadsheet:** stop .xls integer rendering fabricating digits above
2^53 ([#3704](#3704))
([e8a5109](e8a5109))
* **spreadsheet:** stop inventing digits for .xls cells above 2**53
([#3700](#3700))
([1a7c5bb](1a7c5bb))
* **tests:** ask litellm for a model it prices instead of naming a
retired one
([#3722](#3722))
([0ad5e68](0ad5e68))
* **tool-search:** detect every wire shape of client-side deferral
([#3746](#3746))
([665b73d](665b73d))
* **tool-search:** don't orphan deferred tools, and stop overstating the
saving
([#3739](#3739))
([85f9e01](85f9e01))
* **tool-search:** make the core-tools override authoritative and the
warning recurrent
([#3745](#3745))
([a9757c9](a9757c9))
* **transforms:** detect fixed-width command output as tabular and keep
it out of Kompress
([#3786](#3786))
([6c9aef1](6c9aef1))
* **transforms:** stop embedded JSON pre-pass short-circuiting HTML
extraction
([#3620](#3620))
([195910b](195910b))
* **transforms:** stop routing timestamped log rows as grep matches
([#3748](#3748))
([d971f7c](d971f7c))
* **wrap:** handle SIGHUP on the shared proxy paths, not just `claude`
([#3331](#3331))
([c4df2dd](c4df2dd))
* **wrap:** stop blocking the launch on the Serena pre-index
([#3557](#3557))
([df8ebdf](df8ebdf))


### Performance Improvements

* **cache:** drop eager full-backend TTL sweep on every new-key store()
([#3256](#3256))
([2fed685](2fed685))
* **cache:** LRU-bound the compression-feedback tool-pattern map
([#3342](#3342))
([2d10b10](2d10b10))
* **memory:** reindex FTS5 a page at a time instead of a record at a
time ([#3466](#3466))
([3e36aef](3e36aef))
* **providers:** cache Google/Cohere token counters across requests
([#3273](#3273))
([cc6a07c](cc6a07c))
* **relevance:** order BM25 query terms once, not once per document
([#3468](#3468))
([6880984](6880984))


### Dependencies

* bump dotenv from 17.4.2 to 18.0.1 in /sdk/typescript
([#3765](#3765))
([56edfbc](56edfbc))
* bump ruff from 0.16.7 to 0.16.8 in the pip-minor-patch group
([#3761](#3761))
([b9ded1a](b9ded1a))
* bump the cargo-minor-patch group with 3 updates
([#3762](#3762))
([fbbe8b8](fbbe8b8))
* bump the npm-minor-patch group across 2 directories with 19 updates
([#3763](#3763))
([54357ff](54357ff))
* bump the npm-minor-patch group across 3 directories with 9 updates
([#3764](#3764))
([6311332](6311332))


### Code Refactoring

* **sdk:** route _fetch through rawFetch instead of duplicating it
([#3473](#3473))
([b2b5847](b2b5847))
* **smart-crusher:** drop the dead crush_object clone from the object
path ([#3705](#3705))
([7221724](7221724))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ready for review Pull request body is complete and the author marked it ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants