Repository navigation
feat(cli): add headroom inspect to view original vs compressed content - #1595
Conversation
Headroom exposes quantitative compression telemetry (token counts, ratios) but no way to see what the compressor actually changed in the content, which makes it hard to trust compression or debug quality regressions. Add a `headroom diff` command that reads the proxy's existing loopback /transformations/feed endpoint — which already carries the pre/post message snapshots when the proxy runs with --log-messages — and renders, per request, the original vs compressed content for each message with changed segments highlighted (stdlib unified diff, no new dependencies). Supports --last N, --full, and --format json for offline tooling. Closes headroomlabs-ai#1267
PR governanceThis PR follows the template and is marked ready for human review. |
JerrettDavis
left a comment
There was a problem hiding this comment.
The command implementation and tests are a useful direction, but the PR is not wired to the actual CLI command users will run. headroom diff already exists today as the bundled difftastic structural diff command, and on this branch it still wins the top-level command name.
I verified on the PR branch:
python -m headroom.cli --help shows diff Run difftastic (structural diff).
python -m headroom.cli diff --help opens the difftastic help (Usage: difft [OPTIONS] OLD-PATH NEW-PATH), not the new proxy transformations feed command.
The new tests import from headroom.cli.diff import diff and invoke that command object directly, so they do not catch the registration collision. Please either choose a non-conflicting command name, or deliberately move/rename the existing difftastic wrapper and update tests to invoke the real top-level CLI (main) so the shipped command path is covered.
…iff` collision The bundled difftastic wrapper already owns the top-level `diff` command, so the new original-vs-compressed viewer never registered — `headroom diff` still resolved to difftastic. Rename it to `headroom inspect`, which has no collision. Tests now invoke the real top-level CLI (`main`) with the `inspect` subcommand instead of the command object directly, so the shipped registration path is covered and a future name collision would fail the suite. Closes headroomlabs-ai#1267
headroom diff to view original vs compressed contentheadroom inspect to view original vs compressed content
|
@JerrettDavis good catch — the difftastic wrapper owns
Verified locally: |
JerrettDavis
left a comment
There was a problem hiding this comment.
Reviewed the updated headroom inspect version. The previous command-name collision is resolved by registering through the real top-level CLI path, the docs/wiki now name inspect, and CI is green. I also updated the PR body so the description matches the renamed command.
JerrettDavis
left a comment
There was a problem hiding this comment.
Reviewed the updated inspect command implementation and focused tests. I re-ran uv run pytest tests/test_cli_inspect.py -q (7 passed) and uvx [email protected] check headroom/cli/inspect.py tests/test_cli_inspect.py headroom/cli/main.py headroom/cli/__init__.py locally; both pass. The command registration path, error handling, JSON output, and content-diff rendering are covered well enough for this scope.
Description
Headroom exposes plenty of quantitative compression telemetry (token counts, ratios,
headroom perf,/metrics) but no way to actually see what the compressor changed in the content. That makes it hard to trust compression or debug a quality regression ("did it drop something I cared about?").This adds a
headroom inspectcommand (the issue's Option 1). It reads the proxy's existing loopback/transformations/feedendpoint — which already carries the pre/post-compression message snapshots when the proxy runs with--log-messages— and renders, per request, the original vs compressed content for each message with the changed segments highlighted. No new dependencies (stdlibdifflib).Per request it shows the model, per-request token counts + savings, the transforms applied, and a colorized unified diff of each changed message (red = removed, green = added). Clear errors when no proxy is reachable or when the proxy wasn't started with
--log-messages.Side-by-side / interactive rendering (the fuller form of Option 1) can follow as a polish pass; this lands the core "see what changed" capability on data Headroom already captures.
Closes #1267
Type of Change
Changes Made
headroom/cli/inspect.py: newinspectcommand + content-flattening/diff-render helpers.headroom/cli/__init__.py,headroom/cli/main.py: register the command.tests/test_cli_inspect.py: unit tests (content extraction, the no-proxy / no---log-messages/ empty-feed paths, text render, json output).wiki/cli.md: document the command + options.Testing
pytest)ruff check)Test Output
Real Behavior Proof
inspectcommand against a mocked/transformations/feedpayload (one request, a user message with a line removed by SmartCrusher).req-1 gpt-4o,tokens 100 → 40 (saved 60, 60.0%),transforms: SmartCrusher, and a unified diff with the removed line on the original side; no-proxy and missing---log-messagescases raise actionable errors;--format jsonemits the raw feed.--log-messages(the data source — the feed endpoint — is exercised via the mocked payload that mirrors its shape).Review Readiness
Checklist