feat(runtime): CoT-pruning + developer-loop aggregation (#6173)#6239
Conversation
…#6173) Adds two opt-in, default-off context-engineering passes to the compaction layer, implementing the proposal in #6173 by @pavver. Both run only at the compaction boundary, never on the per-turn hot path, so they do not invalidate the provider prompt cache more often than compaction already does. CoT-pruning is gated by `[compaction] strip_reasoning_after_turns` (default 0, disabled). It strips `<think>...</think>` spans and `Thinking` content blocks from assistant turns older than the configured number of assistant turns. It is skipped entirely for models whose `ReasoningEchoPolicy` is `Echo` (DeepSeek V4 Flash), which require the reasoning echoed back on tool_calls turns or the API returns 400. Developer-loop aggregation is gated by `[compaction] aggregate_developer_loops` (default false) with `max_loop_steps_before_aggregate` (default 5). It collapses long runs of consecutive developer-tool steps into the first step, a deterministic placeholder, and the last step. Developer tools are identified by the existing `ToolClass` taxonomy (`Mutating` / `ExecCapable`), not a hardcoded name list. Whole tool-use/result pairs are removed together so no `tool_use_id` is ever orphaned. Both knobs are wired through `CompactionTomlConfig`, the per-agent `CompactionOverrides` in agent.toml, and the runtime `CompactionConfig`, and the kernel-config golden fixture is updated to match. The append-only message journal retains the original messages; only the compacted working session is reshaped.
github-actions
Bot
added
size/L
…by-one Collapse all multi-line ///doc and // block comments in the new context-engineering code to a single line each per the project style rule. Fix `aggregate_developer_loops`: the threshold condition was `steps.len() > max_steps`, so with the default max_steps=5 a run needed 6 steps to trigger — contradicting the field description "minimum N steps before triggers". Change to `>=` so exactly N steps triggers aggregation. Add a boundary test that covers this case.
houko
left a comment
houko
left a comment
There was a problem hiding this comment.
Consecutive Role::User messages in aggregated output (developer_loop_placeholder, compactor.rs)
The placeholder inserted between two elided steps has role: Role::User.
In the aggregated sequence the surrounding layout is:
[tool_use — Assistant] ← first step
[tool_result — User] ← first step result
[placeholder — User] ← ⚠ consecutive User turn
[tool_use — Assistant] ← last step
[tool_result — User]
tool_result and placeholder are both Role::User, producing two consecutive user-role messages.
Most provider APIs (including Anthropic's) accept consecutive user turns in practice — they are coalesced before forwarding — but the Anthropic Messages API rejects two consecutive user messages when they both contain only text blocks if there is no preceding assistant turn (see the "alternating roles" validation in their API docs).
Since compaction is meant to produce a well-formed session, worth checking whether the output of compact_messages is ever fed raw into a provider call without the normal role-coalescing pass.
If the session already runs through a role-merge step before the final API call this is fine; if not, consider making the placeholder Role::Assistant (a brief summary note) or a ContentBlock::ToolResult-style synthetic user block that pairs with a synthetic tool-use so the sequence stays alternating.
Generated by Claude Code
…stfmt find_ascii_ci (#6173)
2 participants
Summary
Implements the two context-engineering features proposed in #6173 by @pavver: CoT-pruning and developer-loop aggregation.
Both are opt-in, default-off passes that run inside the existing compaction pass and reshape only the kept (verbatim) messages.
Features
CoT-pruning —
[compaction] strip_reasoning_after_turns(default0, disabled).Strips
<think>...</think>spans from text and dropsContentBlock::Thinkingblocks from assistant turns older than the configured number of assistant turns.Developer-loop aggregation —
[compaction] aggregate_developer_loops(defaultfalse) +max_loop_steps_before_aggregate(default5).Collapses a run of consecutive developer-tool steps into the first step, a deterministic placeholder, and the last step.
Both knobs are wired through
CompactionTomlConfig(globalconfig.toml), the per-agentCompactionOverrides(agent.toml, refs #5476), and the runtimeCompactionConfig.How the design constraints are satisfied
compactor::compact_messages, never inprepare_llm_messages/ the per-turn hot path, so they do not invalidate the provider prompt cache more often than compaction already does.ReasoningEchoPolicy. CoT-pruning is a no-op forEchomodels (DeepSeek V4 Flash, [Bug] deepseek-v4-flash lacks multi-turn reasoning_content handling — fails after tool calls #4842) — those providers require the reasoning echoed back ontool_callsturns or the API returns 400. The policy is already a parameter ofcompact_messages.tool_use_idis orphaned. Non-loop or short runs are left verbatim.classify_tool→ToolApprovalClass::{Mutating, ExecCapable}, not a hardcoded list (answers the proposal's Q1).Changes
crates/librefang-types/src/config/types.rs— three newCompactionTomlConfigfields + defaults.crates/librefang-types/src/agent.rs— matchingCompactionOverridesfields +resolve/is_empty.crates/librefang-runtime/src/compactor.rs—prune_stale_reasoning,aggregate_developer_loops, helpers, integration intocompact_messages, and unit tests.crates/librefang-api/tests/fixtures/kernel_config_schema.golden.json— regenerated for the three new fields.CHANGELOG.md—[Unreleased]entries.Tests
Unit tests in
compactor.rs: reasoning stripped past the threshold / kept within it,Echo-policy no-op, zero-disabled no-op,<think>strip (case-insensitive / multiple / unclosed), long-run aggregation with tool-pairing preserved, short-run and disabled no-ops, and read-only tools not aggregated.Verification
Local cargo/Docker verification was not run on the authoring host: it has no native Rust toolchain, and the sanctioned Docker dev image compiles into a volume that exhausted the constrained internal disk.
Per the repo's no-local-toolchain fallback, CI is the verification gate here —
cargo fmt, clippy-D warnings, thekernel_config_schemagolden test, andlibrefang-runtime/librefang-typesunit tests all run there.The golden fixture was hand-updated to match schemars output; if CI's golden test reports a diff, I'll reconcile it.
Closes #6173.