Skip to content

fix(transforms): stop routing timestamped log rows as grep matches - #3748

Merged
JerrettDavis merged 3 commits into
headroomlabs-ai:mainfrom
xiechimon:fix/3736-timestamp-prefix-search-guard
Sep 24, 2026
Merged

JerrettDavis merged 3 commits into
headroomlabs-ai:mainfrom
xiechimon:fix/3736-timestamp-prefix-search-guard

Conversation

@xiechimon

@xiechimon xiechimon commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Description

_is_search_result_line now rejects any line that _TIMESTAMP_ROW_RE (from lossless_compaction.py, #3419) recognizes as a timestamped log row, before the grep-shape patterns are tried.

The colon branch accepts ^[^\s:]+:\d+: whenever the prefix passes _prefix_looks_like_path, and that guard only rules out <, > and =. A bare ISO date-hour prefix (2026-09-23T10) passes, so every row of a timestamped log counts as a grep match at ratio 1.0 and the payload routes to SEARCH_RESULTS.

Since #3419 the lossless fold skips timestamp rows, so the payload falls through to the lossy SearchCompressor: 2,000 rows collapse to 5, the note misreads them as "matches in 2026-09-23T10", and the minute is printed back as an integer (10:00:00Z becomes 10:0:00Z, a time that does not exist in the input).

Reusing the lossless module's regex keeps the two guards in lockstep: a row the fold refuses to treat as grep output can no longer be classified as grep output here either. The context-line branch (_is_grep_context_line) already kept dates out via its /-or-. requirement; this closes the same hole in the colon branch.

Closes #3736

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Changes Made

  • headroom/transforms/content_detection.py: _is_search_result_line checks _TIMESTAMP_ROW_RE.match(line) (imported from lossless_compaction) before the colon and context branches, so timestamped log rows are never counted as grep matches.
  • tests/test_transforms/test_content_detector_timestamp_search.py: 6 new regression tests (2 pinning the bug, 4 controls for real grep output, date-named files like logs/2026-09-23.log:42:, and extensionless Makefile:12:).

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom)
  • New tests added for new functionality
  • Manual testing performed

Test Output

Red-green on base 26a2c493 (clean worktree): the two bug-pinning tests fail
(_is_search_result_line returns True; _try_detect_search returns
SEARCH_RESULTS at confidence 1.0); after the fix all 6 pass.

Adjacent suites on this branch:
  tests/test_transforms/                     427 passed, 2 failed
    (test_dense_line_elider, test_smart_crusher_bugs reproduce identically
     on a clean base worktree: pre-existing)
  test_transforms_content_detection.py +
  test_transforms_content_router.py           95 passed
  test_lossless_timestamp_guard.py,
  test_search_compressor*.py,
  test_bash_search_lossless_fold.py           93 passed

ruff check / ruff format --check / mypy: clean on touched files.

Real Behavior Proof

  • Environment: WSL2, Python 3.13.15, editable install of this branch (pure-Python detector; native ONNX path unavailable in this env, same fallback the 0.38.0 reporter's router label shows).
  • Exact command / steps: the 2,000-row ISO-8601 log from the issue, driven through ContentRouter.compress (the same entry headroom_compress uses).
  • Observed result:

Base 26a2c493, before the fix:

detect: ContentType.SEARCH_RESULTS 1.0
strategy: CompressionStrategy.SEARCH
2026-09-23T10:0:00Z INFO worker[0] processed batch 0 in 0ms
2026-09-23T10:0:01Z INFO worker[1] processed batch 1 in 13ms
2026-09-23T10:0:02Z INFO worker[2] processed batch 2 in 26ms
2026-09-23T10:0:03Z INFO worker[3] processed batch 3 in 39ms
2026-09-23T10:33:19Z INFO worker[7] processed batch 1999

That reproduces both reported defects: 5 of 2,000 rows survive, and the minutes are mangled to integers (10:0:00Z).

This branch, after the fix:

detect: ContentType.BUILD_OUTPUT 0.8
strategy: CompressionStrategy.LOG
[2000 lines omitted: 2000 INFO]
[2000 lines compressed to 0. Retrieve more: hash=4803fbc9753df57f2e208836]

The payload now routes as a log, as the issue suggested. No fabricated timestamps appear, and the original stays retrievable through the hash, same as the lossless path.

Runtime Rollout Safety

  • Rollout-managed feature(s): none; this is a pure-Python content-detection guard.
  • Minimum rollout channel: N/A (no feature flag involved).
  • Stable/default behavior changed: only for payloads whose lines match _TIMESTAMP_ROW_RE; those stop being classified as SEARCH_RESULTS and route through the log/build strategies instead. Real grep output is unaffected (controls above).
  • Kill switch / disable path: revert the single guard call in _is_search_result_line.
  • Unsafe override required: no.
  • Qualification impact: none; no qualification suites touched.
  • Rollback path: single-commit revert, no state or schema changes.

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I did not edit CHANGELOG.md — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

Additional Notes

The Rust-side parity gap is intentionally out of scope to avoid conflicting with #3712.

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

PR governance

This PR does not yet satisfy the required template fields:

  • Fill in Real Behavior Proof → Observed result.

Please update the PR body, or move the PR back to draft while it is still in progress.

@github-actions github-actions Bot added the status: needs author action Pull request body or readiness checklist still needs author updates label Sep 23, 2026
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 3b41531 against current main. The timestamp parsing change is correct and well tested, the head is mergeable and contains current main, and all reported checks are terminal green.

@JerrettDavis
JerrettDavis merged commit d971f7c into headroomlabs-ai:main Sep 24, 2026
41 checks passed
JerrettDavis added a commit that referenced this pull request Sep 25, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.39.0](v0.38.0...v0.39.0)
(2026-09-25)


### Features

* **compression:** describe what CCR compression dropped, not just how
much ([#3635](#3635))
([62d1cc0](62d1cc0))
* **hooks:** add protect_messages, a hard per-message compression veto
([#3772](#3772))
([d17ac6a](d17ac6a))
* **opencode:** exclude hosts from transport plugin routing
([#3668](#3668))
([0024b57](0024b57))
* **proxy:** let /transformations/feed omit message bodies for
number-only pollers
([#3672](#3672))
([46ac52d](46ac52d))
* **savings:** carry exact cache-read cost in history rollups
([#3734](#3734))
([1cb779e](1cb779e))
* **savings:** show the new-input savings basis beside the whole-wire
figure
([#3485](#3485))
([e92cccc](e92cccc))
* **savings:** track lifetime output spend so a bill-share rate has a
denominator
([#3520](#3520))
([c766bdb](c766bdb))
* **sdk:** support per-call config override in compress()
([#3370](#3370))
([c3e8a1e](c3e8a1e))
* **wrap:** support 1m context in vscode claude
([#3361](#3361))
([fc5a09e](fc5a09e))


### Bug Fixes

* **anthropic:** reject ping-only SSE from buffered CCR
([#3682](#3682))
([000fefc](000fefc))
* **anthropic:** stop the spurious compaction warning and drop dead
handler code
([#3678](#3678))
([6e71267](6e71267))
* **backends/litellm:** stop dropping image blocks from /v1/messages
requests
([#3753](#3753))
([577336d](577336d))
* **binaries:** fail closed when a download carries no sha256 pin
([#3724](#3724))
([26a2c49](26a2c49))
* **cache/google:** tolerate timezone-aware cache expiry timestamps
([#3210](#3210))
([7ce2580](7ce2580))
* **cache:** default CCR payload previews off; harden the log when they
are on
([#3728](#3728))
([5cb87bc](5cb87bc))
* **cache:** never batch-compress already-forwarded small Responses
outputs in cache mode
([#3756](#3756))
([ab62b9e](ab62b9e))
* **cache:** preserve cache_control schema properties
([#3348](#3348))
([b84c4c9](b84c4c9))
* **cache:** preserve the forwarded OpenAI chat prefix across turns
([#3757](#3757))
([deca575](deca575))
* **cache:** stop prefix transforms from busting the provider cache
([#3735](#3735))
([a2bf5ed](a2bf5ed))
* **cache:** stop skipping tool compaction on a pinned tools array
([#3750](#3750))
([a6a9cef](a6a9cef))
* **cli:** keep non-ASCII working directory names out of the init
profile slug
([#3718](#3718))
([f3d95ee](f3d95ee))
* **cli:** report anonymous beacon status accurately in the banner, log,
and /stats
([#3533](#3533))
([fa9edb3](fa9edb3))
* **content_detector:** detect CMTrace (SCCM/Intune) logs as
BUILD_OUTPUT
([#3712](#3712))
([6feb1fb](6feb1fb))
* **dashboard:** separate rolling cache economics by owner
([#3319](#3319))
([a29162b](a29162b)),
closes [#960](#960)
* **docker:** support OpenCode in native wrapper
([#3547](#3547))
([d90dadf](d90dadf))
* fall back for non-admin Windows task installs
([#3437](#3437))
([b958509](b958509))
* **graph:** fetch and extract the zip codebase-memory-mcp asset on
Windows
([#3730](#3730))
([b776983](b776983))
* **grok:** preserve xAI model context metadata
([#3312](#3312))
([#3315](#3315))
([c81378c](c81378c))
* **init:** raise the ensure hook's timeout above cold-start latency
([#3438](#3438))
([9a11109](9a11109))
* **install:** apply managed env vars added after a deployment was
installed
([#3740](#3740))
([a4cb2bc](a4cb2bc))
* **install:** wait for the old process to stop before restarting
([#3670](#3670))
([11c7320](11c7320))
* **kompress:** bound the inference deadline to the request, not each
block ([#3693](#3693))
([7f2766c](7f2766c))
* **kompress:** keep the boolean connectives that hold a condition
together
([#3687](#3687))
([bf290ba](bf290ba))
* **learn:** treat an unreadable project memory dir as absent instead of
crashing
([#3710](#3710))
([ff60d57](ff60d57))
* **mcp:** accept an empty install ledger instead of failing mutations
as malformed
([#3235](#3235))
([19ddfe1](19ddfe1))
* **memory/graph:** apply relation_type filter to both sides of a BOTH
subgraph query
([#3236](#3236))
([1a6f941](1a6f941))
* **memory:** fail closed for unresolved tools
([#3655](#3655))
([7c3cbc8](7c3cbc8))
* **memory:** make extract_system_prompt concatenative, not first-match
([#3644](#3644))
([3f3cf19](3f3cf19))
* **memory:** tolerate None message content in inline memory parser
([#3211](#3211))
([5ccec67](5ccec67))
* **metrics:** split rate-limited by source and label failures by
provider
([#3707](#3707))
([5ff4ea1](5ff4ea1))
* **openclaw:** declare transcriptSemantics and implement commitTurn
([#3442](#3442))
([a28dd4e](a28dd4e))
* **perf:** make budget checks constant-time
([#3368](#3368))
([ed08069](ed08069))
* **perf:** make rate limiter bucket checks constant-time
([#3373](#3373))
([a8bd9be](a8bd9be))
* preserve null and empty strings in lossless CSV compaction
([#3665](#3665))
([7ad9f80](7ad9f80))
* **proxy:** admit container host gateway on compress routes
([#3715](#3715))
([a8ae0a6](a8ae0a6))
* **proxy:** bound request-log message payloads to the feed window; trim
glibc too
([#3717](#3717))
([994a89f](994a89f))
* **proxy:** count a ceiling-stopped stream exactly instead of
estimating its text
([#3771](#3771))
([b36e59a](b36e59a))
* **proxy:** discard partial SSE tails during finalization
([#3411](#3411))
([6c0e817](6c0e817))
* **proxy:** enforce configured TPM limits
([#3350](#3350))
([138736c](138736c))
* **proxy:** fall back when forced Kompress is cold
([#3381](#3381))
([ca54c2a](ca54c2a))
* **proxy:** make the output shaper actually reduce output in the
default mode
([#3773](#3773))
([c029298](c029298))
* **proxy:** partition response cache by upstream
([#3349](#3349))
([dfdc725](dfdc725))
* **proxy:** pin the validated address so DNS rebinding cannot move an
upstream
([#3731](#3731))
([a102e05](a102e05))
* **proxy:** preserve Claude Code auto-mode protocol
([#3784](#3784))
([12c1579](12c1579))
* **proxy:** preserve empty transport error context
([#3675](#3675))
([ca02f28](ca02f28))
* **proxy:** preserve OpenCode Headroom retrieve results
([#3782](#3782))
([c7ebaed](c7ebaed))
* **proxy:** protect Codex exec reads whose argument is a JavaScript
object literal
([#3737](#3737))
([ecb5e5a](ecb5e5a))
* **proxy:** reject Anthropic batch operations on Copilot
([#3318](#3318))
([871bbde](871bbde))
* **proxy:** repair client-side tool_search tool_reference blocks
([#3443](#3443))
([f12e3fb](f12e3fb))
* **proxy:** resolve Codex custom_tool_call names for Responses tool
exclusion
([#3755](#3755))
([55c78de](55c78de))
* **proxy:** restrict urlopen SSL context ALPN to http/1.1
([#3567](#3567))
([5baa439](5baa439))
* **proxy:** scope OpenAI rate limits by api-key
([#3365](#3365))
([b9e8462](b9e8462))
* **proxy:** stamp x-headroom-* compression metrics on streaming
responses
([#3769](#3769))
([2c4dc44](2c4dc44))
* remove deprecated Apache license classifier
([#3566](#3566))
([b2d9275](b2d9275))
* **savings:** report negative savings instead of flooring them at zero
([#3744](#3744))
([1455f00](1455f00))
* **sdk:** enforce dormant public type tests
([#3424](#3424))
([95cbbb8](95cbbb8))
* **sdk:** preserve Anthropic image blocks in message conversion instead
of dropping the turn
([#3439](#3439))
([63676e6](63676e6))
* **sdk:** preserve proxy authentication alongside provider credentials
([#3450](#3450))
([ca4b47a](ca4b47a))
* **security:** create credential files private instead of narrowing
after write
([#3215](#3215))
([9b8cae8](9b8cae8))
* **security:** reject unauthenticated public proxy binds
([#3316](#3316))
([b0c19a2](b0c19a2))
* **smart-crusher:** disclose dropped scalar-array items with a
retrievable ccr sentinel
([#3692](#3692))
([c1fc84a](c1fc84a))
* **spreadsheet:** stop .xls integer rendering fabricating digits above
2^53 ([#3704](#3704))
([e8a5109](e8a5109))
* **spreadsheet:** stop inventing digits for .xls cells above 2**53
([#3700](#3700))
([1a7c5bb](1a7c5bb))
* **tests:** ask litellm for a model it prices instead of naming a
retired one
([#3722](#3722))
([0ad5e68](0ad5e68))
* **tool-search:** detect every wire shape of client-side deferral
([#3746](#3746))
([665b73d](665b73d))
* **tool-search:** don't orphan deferred tools, and stop overstating the
saving
([#3739](#3739))
([85f9e01](85f9e01))
* **tool-search:** make the core-tools override authoritative and the
warning recurrent
([#3745](#3745))
([a9757c9](a9757c9))
* **transforms:** detect fixed-width command output as tabular and keep
it out of Kompress
([#3786](#3786))
([6c9aef1](6c9aef1))
* **transforms:** stop embedded JSON pre-pass short-circuiting HTML
extraction
([#3620](#3620))
([195910b](195910b))
* **transforms:** stop routing timestamped log rows as grep matches
([#3748](#3748))
([d971f7c](d971f7c))
* **wrap:** handle SIGHUP on the shared proxy paths, not just `claude`
([#3331](#3331))
([c4df2dd](c4df2dd))
* **wrap:** stop blocking the launch on the Serena pre-index
([#3557](#3557))
([df8ebdf](df8ebdf))


### Performance Improvements

* **cache:** drop eager full-backend TTL sweep on every new-key store()
([#3256](#3256))
([2fed685](2fed685))
* **cache:** LRU-bound the compression-feedback tool-pattern map
([#3342](#3342))
([2d10b10](2d10b10))
* **memory:** reindex FTS5 a page at a time instead of a record at a
time ([#3466](#3466))
([3e36aef](3e36aef))
* **providers:** cache Google/Cohere token counters across requests
([#3273](#3273))
([cc6a07c](cc6a07c))
* **relevance:** order BM25 query terms once, not once per document
([#3468](#3468))
([6880984](6880984))


### Dependencies

* bump dotenv from 17.4.2 to 18.0.1 in /sdk/typescript
([#3765](#3765))
([56edfbc](56edfbc))
* bump ruff from 0.16.7 to 0.16.8 in the pip-minor-patch group
([#3761](#3761))
([b9ded1a](b9ded1a))
* bump the cargo-minor-patch group with 3 updates
([#3762](#3762))
([fbbe8b8](fbbe8b8))
* bump the npm-minor-patch group across 2 directories with 19 updates
([#3763](#3763))
([54357ff](54357ff))
* bump the npm-minor-patch group across 3 directories with 9 updates
([#3764](#3764))
([6311332](6311332))


### Code Refactoring

* **sdk:** route _fetch through rawFetch instead of duplicating it
([#3473](#3473))
([b2b5847](b2b5847))
* **smart-crusher:** drop the dead crush_object clone from the object
path ([#3705](#3705))
([7221724](7221724))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: needs author action Pull request body or readiness checklist still needs author updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] 0.38.0: ISO-8601 timestamped logs routed as search results, now folded lossily (5 of 2,000 lines, timestamps rewritten)

3 participants