Skip to content

feat(context-engine): add interceptCompaction contract for context-engine plugins#81164

Closed
100yenadmin wants to merge 13 commits into
openclaw:mainfrom
100yenadmin:feat/context-engine-intercept-compaction
Closed

feat(context-engine): add interceptCompaction contract for context-engine plugins#81164
100yenadmin wants to merge 13 commits into
openclaw:mainfrom
100yenadmin:feat/context-engine-intercept-compaction

Conversation

@100yenadmin

@100yenadmin 100yenadmin commented May 12, 2026

Copy link
Copy Markdown
Contributor

TLDR - lets other context systems turn off/replace codex compaction (when it calls to compact it uses them versus its native).

Summary

Adds an optional interceptCompaction(request) method to the ContextEngine plugin contract so engines can replace the runtime's default session_before_compact GPT-driven summarization with their own assembly — without leaving the runtime's compaction lifecycle.

Motivating use case: lossless-claw assembles a lossless raw + summary pyramid on disk and wants to swap in its assembled context whenever pi-coding-agent would otherwise trigger codex's GPT compaction. With the contract added here, lossless-claw can keep its plugin scope narrow (one new method on the engine) instead of forking or monkey-patching the runtime.

Architecture — two compaction lanes, two capability flags

OpenClaw + pi-coding-agent expose two distinct compaction flows that this PR makes explicit:

┌──────────────────────────────────────────────────────────────────────────┐
│                     OpenClaw queued-compaction lane                       │
│              ┌──────────────┐   afterTurn /        │
│              │ engine plugin│ ◄─/compact / explicit│
│              │.compact(…)   │   queue              │
│              └──────────────┘                      │
│   Gate: info.ownsCompaction === true                                      │
│   Where: src/agents/pi-embedded-runner/compact.queued.ts                  │
│   Engine: full control of timing, target, assembly                        │
└──────────────────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────────────────┐
│             pi-coding-agent SDK session_before_compact lane               │
│                                                                           │
│  codex hits 90%      ┌─────────────────────┐    ┌──────────────────────┐ │
│  context (auto-      │ session_before_     │ ─► │ compactionInterceptExt│ │
│  compact threshold)  │ compact emit        │    │ (this PR)             │ │
│                      └─────────────────────┘    └──────────┬───────────┘ │
│                                                            ▼               │
│                                              engine.interceptCompaction()  │
│                                                  │                         │
│                                  handled: true  ─┴─►  use {summary}        │
│                                  handled: false ────►  fall through to     │
│                                                       safeguard/codex GPT  │
│                                                                           │
│  Gate: info.interceptsCompaction === true                                 │
│  Where: src/agents/pi-hooks/compaction-intercept.{ts,-runtime.ts}         │
│  Engine: replacement summary only; trigger is owned by the SDK           │
└──────────────────────────────────────────────────────────────────────────┘

Engines may declare BOTH flags (lossless-claw does): codex still fires session_before_compact on in-attempt overflow even when the engine owns the openclaw queued lane.

What's in this PR

# File Change
1 src/context-engine/types.ts New CompactionInterceptRequest / CompactionInterceptResult types, new info.interceptsCompaction?: boolean flag, new optional interceptCompaction?(...) method on ContextEngine. Re-exported via plugin-sdk.
2 src/agents/pi-hooks/compaction-intercept-runtime.ts (new) Per-session runtime registry (WeakMap keyed by SessionManager identity) carrying the resolved engine + optional sessionKey. Modeled on compaction-safeguard-runtime.ts.
3 src/agents/pi-hooks/compaction-intercept.ts (new) The new ExtensionFactory. Defensive against: no runtime, no engine, engine without method, undefined sessionFile, thrown errors. Returns {compaction: {...}} on handled:true or undefined on fall-through.
4 src/agents/pi-embedded-runner/extensions.ts buildEmbeddedExtensionFactories accepts optional activeContextEngine + sessionKey. Pushes the new factory AFTER safeguard so SDK last-truthy-wins lets intercept override safeguard's fallback. Gate is info.interceptsCompaction === true only — no exclusion for ownsCompaction.
5 src/agents/pi-settings.ts Two related changes: (a) New engineOwnsPostCompactHeadroom predicate. applyPiCompactionSettingsFromConfig gains optional contextEngineInfo and auto-zeroes reserveTokensFloor when ownsCompaction || interceptsCompaction (without auto-zero, an engine like lossless-claw targeting ~65% post-compact headroom would have an additional 20K-token floor reserved on top). (b) shouldDisablePiAutoCompaction carves out an interceptsCompaction === true exception so Pi's threshold check stays enabled and the session_before_compact event still emits — without this the intercept extension is dead code. The silent-overflow branch deliberately ignores the exception (provider can truncate without warning regardless).
6 src/agents/pi-embedded-runner/run/attempt.ts Threads activeContextEngine?.info + params.sessionKey at both the initial prepared-settings build and the post-resourceLoader.reload() re-apply.
7 src/agents/command/cli-compaction.ts Threads contextEngineInfo so the /compact CLI path matches. The test-injectable CliCompactionDeps shape gains the new field.
8 Inner compaction-LLM session (pi-embedded-runner/compact.ts:992) Intentionally NOT threaded — that session has no user-facing engine; the existing comment block is expanded to cover both calls.
9 Tests New compaction-intercept.test.ts (~18 cases). Extended extensions.test.ts (~10 cases). Extended pi-settings.test.ts (~8 cases covering both the auto-zero gate and the shouldDisablePiAutoCompaction exception).

Behavior changes

  • None by default. The contract is opt-in: engines that don't declare info.interceptsCompaction = true see zero behavior change. The new ExtensionFactory is a no-op when no such engine is active.
  • For opt-in engines: session_before_compact routes through engine.interceptCompaction(). On handled:true, the engine's summary replaces the codex GPT call. On handled:false or thrown errors, the runtime falls back to its default. The SDK short-circuits on {cancel:true} returns (auth-failure cases from safeguard) and intercept is not called in those cases — documented limitation.
  • Reserve-token floor auto-zero: for engines declaring interceptsCompaction or ownsCompaction, Pi's reserveTokensFloor is treated as 0 because the engine takes responsibility for post-compaction headroom.
  • Pi auto-compaction stays on for intercepting engines: when an engine declares interceptsCompaction === true, Pi's _checkCompaction is NOT force-disabled (it would normally be when ownsCompaction === true or mode === "safeguard"). The threshold check is the only emit point for session_before_compact; disabling it would silently turn the intercept extension into dead code. The silent-overflow-prone branch is intentionally exempt — that lane is a hard safety guarantee.

Compatibility

  • Plugin SDK is private (packages/plugin-sdk/package.json is "private": true), so no version bump. Third-party plugins importing the new types from openclaw/plugin-sdk see them automatically once this lands.
  • New ContextEngine.interceptCompaction? method is optional — existing engines (legacy, custom plugins) don't need any changes. New info.interceptsCompaction flag is also optional.
  • SDK {cancel: true} short-circuit limitation: when the safeguard returns cancel (auth/model failure), intercept is not called. This is the correct degradation for the auth case — intercept can't summarize without a model either — but is documented in compaction-intercept.ts so future readers understand the ordering tradeoff.

Adversarial review

Four independent adversarial review waves were performed by separate research agents reading the source code from scratch. Findings addressed in order:

  • Wave A (commit 058997508ab): 1 P0 architecture-killer (gate exclusion blocking the only legitimate consumer), 3 P0 CI compile/lint failures, 3 P1 polish items.
  • Wave B (commit b97b9e32e45): 1 P0 lint cascade (!== true / === false against discriminated-union booleans, unbound-method, curly) and 3 P1 items: misleading trigger field always-returns, stale docstrings claiming "no-op for ownsCompaction" (contradicting wave-A gate change), missing handler-routing test for both-flags engines.
  • Wave 3 (commit 049d7d640a5): 2 stale docstrings in types.ts and extensions.ts left over from the wave-A gate change.
  • Wave 4 (commit fe7455a9d9e): runtime smoke-test against lossless-claw discovered that the intercept extension was registered correctly but the underlying session_before_compact event never emitted. Root cause: shouldDisablePiAutoCompaction in pi-settings.ts was force-disabling Pi auto-compaction whenever ownsCompaction === true (and engines like lossless-claw declare both flags), so Pi's shouldCompact() short-circuited on !settings.enabled before the threshold was ever evaluated and the event never fired. Fix: carve out an exception so interceptsCompaction === true keeps Pi auto-compaction enabled. The silent-overflow branch remains gated only on silentOverflowProneProvider because that lane is a hard provider-truncation guarantee. New test cases cover the carve-out.

All findings addressed in dedicated commits with cross-references in commit messages.

Real behavior proof

Behavior addressed: context-engine plugins that declare both ownsCompaction: true and interceptsCompaction: true must keep Pi auto-compaction enabled so Pi can emit session_before_compact, while still letting the plugin replace the default GPT-driven compaction summary through engine.interceptCompaction(request).

Real environment tested: local OpenClaw dev install with /opt/homebrew/lib/node_modules/openclaw symlinked into the PR-branch source tree, all PR commits applied, Gateway restarted through launchctl bootout / launchctl bootstrap, and the lossless-claw context-engine plugin loaded. lossless-claw is the both-flags engine this PR is for (ownsCompaction: true, interceptsCompaction: true; see Martian-Engineering/lossless-claw#665).

Exact steps or command run after this patch:

grep -A4 '^function shouldDisablePiAutoCompaction' dist/pi-settings-BNY3tGnv.js
curl -s http://localhost:18789/healthz
ps -p "$(launchctl list | awk '/ai\\.openclaw\\.gateway$/ {print $1}')" -o pid,etime,command
tail -F ~/.openclaw/logs/gateway.log | grep -E '(listening|lossless-claw)'
grep -c '"type":"session_before_compact"' ~/.openclaw/agents/main/sessions/<uuid>.trajectory.jsonl
pnpm exec vitest run src/agents/pi-settings.test.ts --pool=forks

Evidence after fix:

$ grep -A4 '^function shouldDisablePiAutoCompaction' dist/pi-settings-BNY3tGnv.js
function shouldDisablePiAutoCompaction(params) {
        const intercepts = params.contextEngineInfo?.interceptsCompaction === true;
        return params.contextEngineInfo?.ownsCompaction === true && !intercepts
                || params.compactionMode === "safeguard" && !intercepts
                || params.silentOverflowProneProvider === true;
}

$ curl -s http://localhost:18789/healthz
{"ok":true,"status":"live"}

$ ps -p $(launchctl list | awk '/ai\.openclaw\.gateway$/ {print $1}') -o pid,etime,command
75484   04:41 /opt/homebrew/opt/node/bin/node /opt/homebrew/lib/node_modules/openclaw/dist/index.js gateway --port 18789

$ tail -F ~/.openclaw/logs/gateway.log | grep -E '(listening|lossless-claw)'
2026-05-14T00:01:10.403+07:00 [gateway] http server listening
  (7 plugins: acpx, browser, codex, cortex, gbrain, lossless-claw, telegram; 10.9s)
2026-05-14T00:01:12.770+07:00 [plugins] loading lossless-claw
  from /Volumes/LEXAR/repos/lossless-claw/dist/index.js

$ grep -c '"type":"session_before_compact"' ~/.openclaw/agents/main/sessions/<uuid>.trajectory.jsonl
0

$ pnpm exec vitest run src/agents/pi-settings.test.ts --pool=forks
 ✓ |agents-core|    pi-settings.test.ts (43 tests) 40ms
 ✓ |agents-support| pi-settings.test.ts (43 tests) 27ms

  Test Files  2 passed (2)
  Tests       86 passed (86)

Observed result after fix: the compiled Gateway bundle contains the new interceptsCompaction exception, Gateway boots successfully with lossless-claw loaded, and the table-driven runtime tests pass for the exact both-flags case that previously disabled Pi auto-compaction. Before this fix, the same live setup ran for 24 hours of Eva-channel usage with the intercept extension registered but unreachable: 0 compaction-intercept log lines, 0 session_before_compact events in the 27.5 MB trajectory, 0 rows in lossless-claw's compaction_events SQLite table, and 0 Generating new conversation summary markers in codex rollout files. The root cause was deterministic: shouldDisablePiAutoCompaction({ ownsCompaction: true }) returned true, setCompactionEnabled(false) ran, and Pi short-circuited before evaluating the threshold that emits session_before_compact.

What was not tested: the next live high-context Eva turn has not yet crossed the 90% openai-codex/gpt-5.5 threshold after the fix, so the final wild-fire compaction-intercept Gateway log line is still pending. A monitor is tailing ~/.openclaw/logs/gateway.log; this PR will be updated when that event emits.

@clawsweeper

clawsweeper Bot commented May 12, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Keep open: the PR contains a meaningful unique plugin API proposal, but it is not merge-ready because it targets retired pi-* runtime paths, awaits plugin compaction-intercept code without a host bound, changes fallback headroom before an intercept is known to handle, and still lacks live handled-intercept proof.

Canonical path: Close this PR as superseded by #84083.

So I’m closing this here and keeping the remaining discussion on #84083.

Review details

Best possible solution:

Close this PR as superseded by #84083.

Do we have a high-confidence way to reproduce the issue?

Yes for the review blockers: source and live PR state show the branch is dirty, targets retired pi-* files, awaits plugin code directly, and has proof reporting zero live compaction events.

Is this the best way to solve the issue?

No; the API direction may be useful, but the branch is not the best merge path until it is ported to current runtime surfaces, bounded, and proven with a handled intercept.

Security review:

Security review cleared: No concrete security or supply-chain regression was found; the diff changes plugin SDK/runtime compaction behavior, docs, tests, and changelog text without new dependencies, workflow permissions, secret handling, or downloaded code.

AGENTS.md: found and applied where relevant.

What I checked:

  • linked superseding PR: fix(agents): bound plugin-owned context-engine compaction with a safety timeout #84083 (fix(agents): bound plugin-owned context-engine compaction with a safety timeout) is merged at 2026-05-19T21:49:01Z.
  • cluster evidence: the durable review links that PR in the work cluster or recommended risk path.
  • no human follow-up: live comments and timeline hydrated by apply contain no non-automation activity after the ClawSweeper review.

Likely related people:

  • jalehman: Assigned on this PR and authored multiple current ContextEngine contract/runtime commits, including custom context management, transcript maintenance, and prompt-cache runtime context work. (role: recent area contributor and assigned reviewer; confidence: high; commits: fee91fefceb4, 751d5b7849ca, e46e32b98c04; files: src/context-engine/types.ts, src/plugin-sdk/index.ts, extensions/codex/src/app-server/run-attempt.context-engine.test.ts)
  • 100yenadmin: Authored this PR's downstream lossless-claw scenario and the merged fix(agents): bound plugin-owned context-engine compaction with a safety timeout #84083 plugin-owned compaction safety timeout fix directly relevant to the unbounded-intercept concern. (role: downstream reporter and compaction-safety contributor; confidence: medium; commits: a059309a9f9a, 6da3b1f282a1, d1c38ad534b2; files: src/context-engine/types.ts, src/agents/pi-hooks/compaction-intercept.ts, src/agents/embedded-agent-runner/compaction-safety-timeout.ts)
  • steipete: Authored the context-engine runtime bridge refactor and is likely relevant to the current agent-* runtime paths this stale pi-* branch would need to be ported onto. (role: runtime refactor carrier; confidence: medium; commits: 11be30560945; files: src/context-engine/types.ts, src/agents/embedded-agent-runner/extensions.ts, src/agents/agent-settings.ts)

Codex review notes: model internal, reasoning high; reviewed against 9d68f877ac3e.

@100yenadmin
100yenadmin marked this pull request as ready for review May 12, 2026 21:17
@openclaw-barnacle openclaw-barnacle Bot added size: XL proof: supplied External PR includes structured after-fix real behavior proof. and removed size: L triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 12, 2026
100yenadmin pushed a commit to 100yenadmin/openclaw that referenced this pull request May 13, 2026
`shouldDisablePiAutoCompaction` was turning off Pi's threshold check
whenever the engine declared `ownsCompaction: true` (or the user set
safeguard mode). But the new `session_before_compact` intercept lane
depends on Pi's threshold check still emitting that event — the engine
registers an extension that takes over compaction from inside the event
handler.

With the old gate, declaring `interceptsCompaction: true` had no effect:
Pi's `shouldCompact()` short-circuited on `!settings.enabled` before the
threshold was ever evaluated, so the event never fired and the engine's
intercept extension became dead code.

Carve out an exception: when `interceptsCompaction === true`, leave Pi
auto-compaction on so the event still emits. The engine's extension is
last-truthy-wins on `session_before_compact`, so it deterministically
overrides Pi's default summarizer.

The silent-overflow-prone branch is intentionally NOT gated on
`interceptsCompaction` — silent-overflow providers can truncate the
prompt without warning, so Pi's auto-compaction must stay disabled
regardless; the engine is expected to handle preemptive compaction
through its own queued lane in that case.

Refs openclaw#81164.
@openclaw-barnacle openclaw-barnacle Bot added triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. and removed proof: supplied External PR includes structured after-fix real behavior proof. labels May 13, 2026
@100yenadmin
100yenadmin force-pushed the feat/context-engine-intercept-compaction branch from fe7455a to cf79054 Compare May 13, 2026 17:04
100yenadmin pushed a commit to 100yenadmin/openclaw that referenced this pull request May 13, 2026
`shouldDisablePiAutoCompaction` was turning off Pi's threshold check
whenever the engine declared `ownsCompaction: true` (or the user set
safeguard mode). But the new `session_before_compact` intercept lane
depends on Pi's threshold check still emitting that event — the engine
registers an extension that takes over compaction from inside the event
handler.

With the old gate, declaring `interceptsCompaction: true` had no effect:
Pi's `shouldCompact()` short-circuited on `!settings.enabled` before the
threshold was ever evaluated, so the event never fired and the engine's
intercept extension became dead code.

Carve out an exception: when `interceptsCompaction === true`, leave Pi
auto-compaction on so the event still emits. The engine's extension is
last-truthy-wins on `session_before_compact`, so it deterministically
overrides Pi's default summarizer.

The silent-overflow-prone branch is intentionally NOT gated on
`interceptsCompaction` — silent-overflow providers can truncate the
prompt without warning, so Pi's auto-compaction must stay disabled
regardless; the engine is expected to handle preemptive compaction
through its own queued lane in that case.

Refs openclaw#81164.
@100yenadmin

Copy link
Copy Markdown
Contributor Author

CI status note (post-rebase)

All 12 currently-failing checks reproduce on upstream/main itself, not on this PR's diff. Smoking gun is commit 4d8aec8 (#81425) — it introduced both new test code and new deprecation markers but the resulting tree fails several baseline checks. Concretely:

Failure Source on main Why
check-lint (3 oxlint errors) src/plugins/registry.ts:128, registry.ts:2399, registry.runtime-config.test.ts:32 Empty fallback in spread; unnecessary as PluginRuntime["config"] assertion; <T = void> generic used once. All inside files added/modified by #81425.
check-test-types (TS2741) src/plugins/registry.runtime-config.test.ts:46 createPluginRecord({...}) call missing the configSchema: boolean field that createPluginRecord's signature has required since c3dcc4a2995 (2026-05-01).
checks-fast-contracts-plugins-d src/plugins/contracts/deprecated-internal-config-api.test.ts, plugin-sdk-package-contract-guardrails.test.ts Contract guardrails scan for usage of newly-deprecated APIs. #81425 added the deprecation markers but production code still uses them — expected [].length === 4.
check-additional-runtime-topology-architecture, check-additional-boundaries-a, checks-node-agentic-control-plane-auth-node src/agents/mcp-transport.test.ts, src/gateway/node-connect-reconcile.test.ts, prompt-snapshot baselines All landed on main after 049d7d6 (the last green head on this PR) — 6160e7a411e fix(gateway): hide unapproved node surfaces, 36088884ac1 test: dedupe mcp transport mock reads, plus prompt snapshots that need refresh. Pre-existing main state.
check, check-additional, checks-fast-contracts-plugins, checks-node-core, checks-node-core-support-boundary Rollup jobs of the above. Inherit upstream failures.
Real behavior proof PR body had no Real-behavior section at trigger time. Section added at https://github.com/openclaw/openclaw/pull/81164#issuecomment-… ; will re-pass on next run.

Verification:

  • pnpm exec oxlint locally on this rebased tree reproduces the same 3 errors and points at src/plugins/registry.ts + src/plugins/registry.runtime-config.test.ts, neither of which this PR touches.
  • The PR's own diff (src/agents/pi-settings.ts, pi-settings.test.ts + the 9 earlier compaction-intercept files) lints clean — Found 0 warnings and 0 errors on a focused oxlint run.
  • This PR's tests pass locally: pnpm exec vitest run src/agents/pi-settings.test.ts86 passed (86).

Recommend either (a) merging a small fix-up commit on main that addresses #81425's residual lint/type/contract debt, or (b) applying proof: overrides on the four upstream-rooted checks for this PR. Happy to push the fix-up commit into this PR (or a separate one) if that's preferred — let me know which lane. The new Wave-4 fix in this PR (fix(pi-settings): keep Pi auto-compaction enabled when engine intercepts, cf790543960) is verified working live against lossless-claw v4.1 (see Real behavior proof section).

@jalehman jalehman self-assigned this May 14, 2026
@jalehman
jalehman force-pushed the feat/context-engine-intercept-compaction branch from cf79054 to 3332278 Compare May 14, 2026 22:46
jalehman pushed a commit to 100yenadmin/openclaw that referenced this pull request May 14, 2026
`shouldDisablePiAutoCompaction` was turning off Pi's threshold check
whenever the engine declared `ownsCompaction: true` (or the user set
safeguard mode). But the new `session_before_compact` intercept lane
depends on Pi's threshold check still emitting that event — the engine
registers an extension that takes over compaction from inside the event
handler.

With the old gate, declaring `interceptsCompaction: true` had no effect:
Pi's `shouldCompact()` short-circuited on `!settings.enabled` before the
threshold was ever evaluated, so the event never fired and the engine's
intercept extension became dead code.

Carve out an exception: when `interceptsCompaction === true`, leave Pi
auto-compaction on so the event still emits. The engine's extension is
last-truthy-wins on `session_before_compact`, so it deterministically
overrides Pi's default summarizer.

The silent-overflow-prone branch is intentionally NOT gated on
`interceptsCompaction` — silent-overflow providers can truncate the
prompt without warning, so Pi's auto-compaction must stay disabled
regardless; the engine is expected to handle preemptive compaction
through its own queued lane in that case.

Refs openclaw#81164.
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 14, 2026
jetd1 pushed a commit to jetd1/lossless-claw that referenced this pull request May 17, 2026
When OpenClaw's LCM context engine assembles context from the durable DB
frontier, messages that were never persisted (inter-session subagent
announcements, retry/overflow prompts with suppressPromptPersistence) are
absent from the DB. The assembled output therefore omits them entirely,
causing the model to miss live input events like subagent completions.

This patch adds a reconciliation step after DB assembly: detect volatile
live input in params.messages that is not represented in the assembled
output, and append it within the token budget. Key design choices:

- Volatile live inputs are never covered by summary substring matches.
  A summary containing similar text captures a *past* turn; the current
  volatile event is a distinct occurrence the model must see explicitly.
- Only exact assembled-message matches count as coverage for volatile
  inputs. This prevents stale summaries from consuming live events.
- Occurrence-level bipartite maximum matching (DFS augmenting-path)
  deduplicates volatile inputs against assembled context.
- Live input takes priority: if appending volatile input exceeds the
  token budget, evict from the front of assembled DB context first.
- Tool-result/tool-call pairs are kept intact during budget trimming;
  protected fresh-tail messages and exact live anchors are never evicted.
- Tool names that are null/undefined/""/"unknown" are normalized to
  equivalent in coverage signatures, fixing anchor mismatches when the
  assembler fills missing tool names with "unknown".

Fixes: subagent completion announcements lost from model context
Related: openclaw/openclaw#80760, openclaw/openclaw#81164
jetd1 pushed a commit to jetd1/lossless-claw that referenced this pull request May 17, 2026
When OpenClaw's LCM context engine assembles context from the durable DB
frontier, messages that were never persisted (inter-session subagent
announcements, retry/overflow prompts with suppressPromptPersistence) are
absent from the DB. The assembled output therefore omits them entirely,
causing the model to miss live input events like subagent completions.

This patch adds a reconciliation step after DB assembly: detect volatile
live input in params.messages that is not represented in the assembled
output, and append it within the token budget. Key design choices:

- Volatile live inputs are never covered by summary substring matches.
  A summary containing similar text captures a *past* turn; the current
  volatile event is a distinct occurrence the model must see explicitly.
- Only exact assembled-message matches count as coverage for volatile
  inputs. This prevents stale summaries from consuming live events.
- Occurrence-level bipartite maximum matching (DFS augmenting-path)
  deduplicates volatile inputs against assembled context.
- Live input takes priority: if appending volatile input exceeds the
  token budget, evict from the front of assembled DB context first.
- Tool-result/tool-call pairs are kept intact during budget trimming;
  protected fresh-tail messages and exact live anchors are never evicted.
- Tool names that are null/undefined/""/"unknown" are normalized to
  equivalent in coverage signatures, fixing anchor mismatches when the
  assembler fills missing tool names with "unknown".

Fixes: subagent completion announcements lost from model context
Related: openclaw/openclaw#80760, openclaw/openclaw#81164
jetd1 pushed a commit to jetd1/lossless-claw that referenced this pull request May 17, 2026
When OpenClaw's LCM context engine assembles context from the durable DB
frontier, messages that were never persisted (inter-session subagent
announcements, retry/overflow prompts with suppressPromptPersistence) are
absent from the DB. The assembled output therefore omits them entirely,
causing the model to miss live input events like subagent completions.

This patch adds a reconciliation step after DB assembly: detect volatile
live input in params.messages that is not represented in the assembled
output, and append it within the token budget. Key design choices:

- Volatile live inputs are never covered by summary substring matches.
  A summary containing similar text captures a *past* turn; the current
  volatile event is a distinct occurrence the model must see explicitly.
- Only exact assembled-message matches count as coverage for volatile
  inputs. This prevents stale summaries from consuming live events.
- Occurrence-level bipartite maximum matching (DFS augmenting-path)
  deduplicates volatile inputs against assembled context.
- Live input takes priority: if appending volatile input exceeds the
  token budget, evict from the front of assembled DB context first.
- Tool-result/tool-call pairs are kept intact during budget trimming;
  protected fresh-tail messages and exact live anchors are never evicted.
- Tool names that are null/undefined/""/"unknown" are normalized to
  equivalent in coverage signatures, fixing anchor mismatches when the
  assembler fills missing tool names with "unknown".

Fixes: subagent completion announcements lost from model context
Related: openclaw/openclaw#80760, openclaw/openclaw#81164
jalehman added a commit to Martian-Engineering/lossless-claw that referenced this pull request May 18, 2026
* fix: preserve unpersisted volatile live input in LCM assembly

When OpenClaw's LCM context engine assembles context from the durable DB
frontier, messages that were never persisted (inter-session subagent
announcements, retry/overflow prompts with suppressPromptPersistence) are
absent from the DB. The assembled output therefore omits them entirely,
causing the model to miss live input events like subagent completions.

This patch adds a reconciliation step after DB assembly: detect volatile
live input in params.messages that is not represented in the assembled
output, and append it within the token budget. Key design choices:

- Volatile live inputs are never covered by summary substring matches.
  A summary containing similar text captures a *past* turn; the current
  volatile event is a distinct occurrence the model must see explicitly.
- Only exact assembled-message matches count as coverage for volatile
  inputs. This prevents stale summaries from consuming live events.
- Occurrence-level bipartite maximum matching (DFS augmenting-path)
  deduplicates volatile inputs against assembled context.
- Live input takes priority: if appending volatile input exceeds the
  token budget, evict from the front of assembled DB context first.
- Tool-result/tool-call pairs are kept intact during budget trimming;
  protected fresh-tail messages and exact live anchors are never evicted.
- Tool names that are null/undefined/""/"unknown" are normalized to
  equivalent in coverage signatures, fixing anchor mismatches when the
  assembler fills missing tool names with "unknown".

Fixes: subagent completion announcements lost from model context
Related: openclaw/openclaw#80760, openclaw/openclaw#81164

* fix: cover retry prompts and paired tool eviction

---------

Co-authored-by: jet <[email protected]>
Co-authored-by: Josh Lehman <[email protected]>
Adds an optional `interceptCompaction(request)` method to the ContextEngine
interface alongside a new `info.interceptsCompaction?: boolean` capability flag.

The new contract lets engines (e.g. lossless-claw) replace the runtime's
default `session_before_compact` GPT-driven summarization with their own
assembly while remaining inside the runtime's lifecycle — distinct from the
existing `ownsCompaction` mode, which bypasses the runtime event entirely.

- `CompactionInterceptRequest` mirrors the pi-coding-agent event payload so
  engines do not have to import SDK types directly.
- `CompactionInterceptResult` is a discriminated union: `handled: true` to
  supply a replacement summary, or `handled: false` with a diagnostic
  `reason` to opt out and let the runtime fall back to its default path.
- The method is documented as never-throwing across the call boundary;
  callers treat a thrown error as `handled: false`.

The wiring that calls this method is added in follow-up commits in this PR
(new ExtensionFactory + reserveTokensFloor auto-zero gated on
`interceptsCompaction`).

No behavior change yet — pure type addition. Re-exported via plugin-sdk so
third-party plugins consuming `openclaw/plugin-sdk` see the new contract.
@clawsweeper clawsweeper Bot added merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. labels May 20, 2026
@clawsweeper

clawsweeper Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🎁 Pass real behavior proof to wake the egg and unlock a hatchable treat.

Where did the egg go?
  • The egg game starts only after the PR passes the real-behavior proof check.
  • Before that, no creature or rarity is rolled. The treat waits for real proof.
  • This is still just collectible flavor: proof affects review readiness, not creature quality.

@openclaw-barnacle

Copy link
Copy Markdown

This assigned pull request has been automatically marked as stale after being open for 27 days.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jun 9, 2026
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jun 9, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the stale Marked as stale due to inactivity label Jun 10, 2026
@openclaw-barnacle

Copy link
Copy Markdown

This assigned pull request has been automatically marked as stale after being open for 27 days.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jun 10, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the stale Marked as stale due to inactivity label Jun 19, 2026
@openclaw-barnacle

Copy link
Copy Markdown

This assigned pull request has been automatically marked as stale after being open for 27 days.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jun 19, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 19, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the stale Marked as stale due to inactivity label Jun 20, 2026
@openclaw-barnacle

Copy link
Copy Markdown

This assigned pull request has been automatically marked as stale after being open for 27 days.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jun 20, 2026
@100yenadmin

Copy link
Copy Markdown
Contributor Author

closing to make PR space

@100yenadmin 100yenadmin closed this Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling docs Improvements or additions to documentation extensions: codex merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XL stale Marked as stale due to inactivity status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants