Skip to content

Compaction tokens are billed but never counted: run usage omits every responses.compact call #4434

Closed

Description

@arthi-arumugam-git

OpenAIResponsesCompactionSession.run_compaction makes a real, billed model call and then drops its usage on the floor.

compacted = await self.client.responses.compact(**compact_kwargs)

compacted = await self.client.responses.compact(**compact_kwargs)

output_items = _strip_orphaned_assistant_ids(
    _normalize_compaction_output_items(compacted.output or [])
)

Only compacted.output is read. compacted.usage is a required field on the response, documented as token accounting for the compaction pass, and the string usage does not appear anywhere in that module. Those tokens never reach context_wrapper.usage, so RunResult reports a total that excludes them, and the docs say usage is aggregated across all model calls during the run.

Why it matters

Compaction runs on long conversations, which is exactly where the compacted history is large, so the missing call is often the most expensive one in the run. In a scripted 12-turn run with one compaction pass, the run reports requests=12 and total_tokens=1800 while the compaction pass itself consumed 192,000 tokens. The reported total is under one percent of what was billed, and requests undercounts by one per pass.

Direction of the error is always the same: reported usage is too low, and it silently diverges further the longer the conversation runs. Nothing raises.

Reproduction

No network or API key. Construct the session with a stubbed client whose responses.compact returns a response carrying usage, run a compaction pass, and inspect any usage the session exposes: there is none, and context_wrapper.usage from the run is unchanged.

Duplicate check

I read the six open and closed compaction issues (#2244, #2317, #2333, #2343, #2727, #3116) and the twelve compaction PRs before filing; they cover latency, ordering, stateless mode, reasoning ids and history loss, none of them token accounting. git log -S "compacted.usage" is empty, and none of the seven open PRs touches this path. Every existing compaction test stubs the response with MagicMock() and sets only .output, so nothing asserts the current behavior.

Fix

A PR follows immediately: record the compaction pass on the session so the tokens are visible instead of discarded, with the accounting written defensively so it can never break a run. Whether that total should also flow into RunResult.context_wrapper.usage automatically is an architectural call I would rather leave to you, since the session has no reference to the run context today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions