Skip to content

test(worker): pin one assistant turn per observer reply, summary included - #3619

Merged
thedotmack merged 1 commit into
thedotmack:mainfrom
ntdatt812:fix/openai-compatible-history-double-append
Sep 30, 2026
Merged

thedotmack merged 1 commit into
thedotmack:mainfrom
ntdatt812:fix/openai-compatible-history-double-append

Conversation

@ntdatt812

@ntdatt812 ntdatt812 commented Aug 17, 2026 •

Copy link
Copy Markdown
Contributor

Status

The source fix from this PR is on main. 7a7ada5 ("bound observer conversation growth…", on main since 2026-08-31) removed the provider-side conversationHistory.push calls, and its message cites this PR. This branch is now rebased onto current main and cut down to the test file. It no longer touches src/.

The one assistant push left in OpenAICompatibleProvider is in handleInitResponse, and it is not a double append. Since a00ce61 the init reply no longer goes through processAgentResponse, so the provider records it itself, once. processAgentResponse records the observation and summary replies.

Why the test is still worth having

main pins the observation path: tests/worker/openai-compatible-init-response.test.ts asserts that roles alternate after init plus one observation.

Two paths have no test:

  • Summary. Putting the push back in processSummaryMessage changes no result in any of the 21 test files that drive the OpenAI-compatible providers or processAgentResponse.
  • Empty observation reply, not forwarded. When a provider does not forward empty replies (forwardEmptyMessageResponse = false), an empty observation reply must add no assistant turn. Pushing one there also changes no result in those 21 files.

This file pins both:

  • records each reply exactly once across the init, observation and summary turns: three queries, roles alternate user, assistant three times, and the assistant turns are the three replies in order.
  • adds no assistant turn for an empty observation reply when empty replies are not forwarded: roles are user, assistant, user.

Verification

Each mutation below was applied to main's OpenAICompatibleProvider.ts. The 21 upstream files were compared with an unmutated run by test name, because this Windows machine has pre-existing failures in that set.

Mutation 21 upstream files This file
Put the push back in processObservationMessage init-response fails test 1 fails
Put the push back in processSummaryMessage no change test 1 fails
Push an empty turn for an unforwarded empty observation reply no change test 2 fails

The new file passes (2 pass, 0 fail). It also type-checks clean against the root tsconfig.json (strict) when included explicitly; tests/ is excluded from the regular typecheck script.

Refs #3499, #3497. Tracked under #3606 (plan-18).

@ntdatt812
ntdatt812 force-pushed the fix/openai-compatible-history-double-append branch from df472be to 569ff25 Compare August 17, 2026 08:26
@greptile-apps

greptile-apps Bot commented Aug 17, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

Safe to merge.

Summary

This PR adds regression coverage for OpenAI-compatible provider conversation history. The tests verify that init, observation, and summary replies are recorded once, roles alternate correctly, and empty observations do not create assistant turns.

Reviews (2) · Last reviewed commit: "test(worker): pin one assistant turn per..."

tt12b pushed a commit to tt12b/claude-mem that referenced this pull request Sep 11, 2026
…d request loops

Three users were forced onto higher-tier plans by the observer burning their
subscription quota. Two independent unbounded loops are responsible.

1. Quadratic conversation growth, then a permanent wedge (thedotmack#3800)

The observer keeps one long-lived conversation per session and appends every
captured tool call to it. Nothing trimmed it, and OpenAICompatibleProvider
re-sent the whole array on every observation, so total input tokens for a
session were O(N^2) in the number of tool calls. A single observation prompt is
field-truncated to ~32k chars, so ~25-30 observations crossed a 200k ceiling.

Past that the provider answers "Prompt is too long" as ordinary assistant text.
It was not classified, so it fell through to generic prose, the batch was
confirmed and dropped, and the next observation appended yet more — every later
tool call paid full freight for a request that could never succeed. A reporter
logged 2,264 of these in one day across 28 sessions.

Nothing stopped it: consecutiveInvalidOutputs was assigned 0 at all five sites
and never incremented, so no breaker keyed on it could trip, and the rejection
log line printed the counter it had just zeroed — which read as "breaker fine"
on every failure.

  - Add compactConversationHistory: pins the framing message, keeps the newest
    turns, drops the middle oldest-first, and announces the elision. Per-request
    size is now flat in session length instead of linear.
  - Add isContextOverflowObserverOutput and handle it: preserve the batch,
    recycle the conversation, force a fresh one, and hard-pause after two
    consecutive failures instead of retrying forever.
  - Stop appending rejections to history — a failure must not enlarge the next
    request.
  - Stop double-appending assistant replies (provider + ResponseProcessor both
    pushed the same text, inflating every later request by ~50%) (thedotmack#3619).
  - Count overflows on their own counter so benign skip acknowledgements keep
    resetting without rebuilding the respawn debt that reset was added to stop.

2. One doomed request per observation after the allowance is spent (thedotmack#3634)

Once a provider reported the inference allowance exhausted, the generator exited
and the very next captured tool call started a fresh one that earned the same
refusal — for the rest of the billing cycle. 11 capped users produced tens of
thousands of cap events in a day, ~100 per successful observation.

ensureGeneratorRunning already had this gate for a missing Claude CLI; add the
same shape for quota. The breaker arms on a quota exit (classified error or
quota prose), withholds requests for a cooldown, lets exactly one probe through
after it, and clears immediately on a successful store.

Full suite green: 2743 pass, 0 fail across 256 files.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01ERvWLxiY96C7pJrx7hPjh8
@thedotmack

Copy link
Copy Markdown
Owner

Routing to #3606 plan-18 (Observer Response Pipeline) — "conversationHistory unbounded / double-appended" is listed in the master, and this is the double-append half.

Verified on main: OpenAICompatibleProvider.ts:234 pushes the assistant reply and ResponseProcessor.ts:311 pushes it again, so OpenRouter and Gemini really do send user, assistant, assistant, ... upstream while ClaudeProvider does not. Leaving open — this is the cleanest fix in the cluster at +168/-3 and should land ahead of the larger context work in #3625, which duplicates it as a side effect.

…uded

7a7ada5 removed the provider-side pushes that recorded every reply twice.
processAgentResponse now records the observation and summary replies, and
handleInitResponse records the init reply, which since a00ce61 no longer
reaches processAgentResponse.

The observation path is pinned: openai-compatible-init-response.test.ts
asserts that roles alternate. The summary path is not. Putting the push
back in processSummaryMessage changes no result in the 21 test files that
drive the OpenAI-compatible providers or processAgentResponse, and neither
does pushing an empty turn when an empty observation reply is not
forwarded.

Pin both: init, observation and summary each leave exactly one assistant
turn and the roles alternate, and an unforwarded empty observation reply
adds none.
@ntdatt812
ntdatt812 force-pushed the fix/openai-compatible-history-double-append branch from 569ff25 to 59cd021 Compare September 19, 2026 02:45
@ntdatt812 ntdatt812 changed the title fix(worker): stop double-appending assistant replies to observer history test(worker): pin one assistant turn per observer reply, summary included Sep 19, 2026
@ntdatt812

Copy link
Copy Markdown
Contributor Author

Rebased onto current main and narrowed to tests: the source change is already on main via 7a7ada5. The assistant push still in OpenAICompatibleProvider is handleInitResponse's. Since a00ce61 the init reply no longer reaches processAgentResponse, so it is recorded once. What main still lacks is a test for the summary path and for an unforwarded empty observation reply. Putting either regression back changes no result in the 21 related test files, and this file catches both. The mutation table is in the description.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The test-only changes provide focused regression coverage with no unresolved issues.

Review effort: Lite
Findings: None

What changed in this PR

Adds regression tests for OpenAI-compatible conversation-history bookkeeping.

Changes:

  • Verifies replies are appended exactly once across init, observation, and summary turns.
  • Verifies unforwarded empty replies add no assistant turn.
File Description
tests/​worker/​openai-compatible-history-append.test.ts Adds conversation-history append regression tests.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@thedotmack
thedotmack merged commit b96cac3 into thedotmack:main Sep 30, 2026
7 checks passed
@thedotmack

Copy link
Copy Markdown
Owner

Merged, thank you @ntdatt812! These tests pin the rule that every observer reply lands in the conversation history exactly once, covering the summary turn and the empty-reply case that main's existing tests never touched. The duplicate-append fix on main already cited this PR, and now there's a regression net behind it. Your mutation table made the value easy to see.

thedotmack added a commit to matthewdnye/claude-mem that referenced this pull request Sep 30, 2026
… cap configurable, name truncations

Trimmed to the three pieces main still lacked, rebuilt on main's request path
(buildOpenRouterRequestBody, the thedotmack#4003 max_completion_tokens retry, the
trial-expiry fallback clearing and the gateway classification stay as thedotmack#4276
left them):

- System anchor: a generation's framing prompt (marked by
  openObserverGeneration) goes out as the system message on OpenRouter and
  custom endpoints and as systemInstruction on Gemini. Its user-request block
  (or a fixed kickoff) stays the first user turn, so every request, the init
  one included, carries a user message and roles still alternate. Standalone
  requests (field condensation, the wrap-up) are unchanged.
- CLAUDE_MEM_OBSERVER_MAX_OUTPUT_TOKENS (default 4096, 256 to 1,000,000):
  max_tokens in buildOpenRouterRequestBody, max_completion_tokens on the
  compatibility retry, Gemini's maxOutputTokens, and the cap thedotmack#3620's cut-off
  warning names. Validated on the settings API and documented.
- finishReason on ProviderQueryResult for OpenRouter and Gemini, a Gemini
  MAX_TOKENS warning for parity, and finishReason/truncated on the drop log
  line. The reason is scoped to one reply: processAgentResponse consumes and
  clears it. The invalid-output counter and the preserve policy stay as they
  are (C05).

Dropped as superseded: the sliding window and its message/char caps (main's
generation recycle and window-aware budget, thedotmack#3625), the attempt timeout and
timeout:false (CLAUDE_MEM_LLM_TIMEOUT_MS), the per-provider
CLAUDE_MEM_OPENROUTER_MAX_TOKENS (one shared setting), the duplicate history
test (thedotmack#3619) and the counter change.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01697ztU3scSYRfW49cPr9Du
thedotmack added a commit that referenced this pull request Sep 30, 2026
…put cap configurable (#3868)

* fix(openrouter): bound the observer context and anchor the schema in a system message (#3606)

The OpenAI-compatible path sent the entire per-session conversation history on
every call with no system role. Against a bounded local context (Ollama,
llama.cpp) that either fills the window (truncation → empty content) or
scrolls the observation schema — stated once in the init prompt — out of
view, the model degrades to bare prose. Both end in ResponseProcessor's
invalid-parse branch, which confirmed and dropped the batch, and the
write-only consecutiveInvalidOutputs counter (always reset to 0) hid how
often this was happening.

Root cause and fix:
- context-window.ts (new): history[0] (the init/continuation prompt, which
  carries the full <observation> schema) is pinned as `system`; the rest is
  windowed by turn count and total chars, the final message is always kept,
  and leading assistant turns are dropped after trimming.
- Four settings now declared and honored on the OpenRouter path:
  CLAUDE_MEM_OPENROUTER_MAX_TOKENS (was hard-coded 4096),
  _MAX_CONTEXT_MESSAGES, _MAX_CONTEXT_CHARS (0 = unbounded),
  _ATTEMPT_TIMEOUT_MS (was the hard-coded 30s withRetry default). A
  corrupt/negative value falls back to the shipped default; max_tokens and
  the timeout require >= 1, the two context caps keep 0 as "unbounded".
- Bun's fetch has a built-in ~300s timeout that overrides withRetry's
  AbortSignal-based per-attempt timeout, so the request init now sets
  `timeout: false` explicitly (Bun-specific, not in the DOM RequestInit
  type — cast through Record<string, unknown>).
- finish_reason is read and propagated (ProviderQueryResult.finishReason,
  session.lastFinishReason). The empty-content branch is now three-way: no
  choices/message at all is `malformed` (a different failure than a designed
  empty completion); `finish_reason === 'length'` is a real truncation
  (error); `=== 'stop'` is the prompt's documented empty-reply skip (info);
  anything else — content_filter, tool_calls, an absent reason — is neither,
  and is not a benign skip (warn).
- ResponseProcessor.ts's generic non-XML fall-through branch (shared by
  every provider — Claude/SDK, Gemini, OpenRouter) now: increments
  consecutiveInvalidOutputs on any real dropped batch instead of leaving it
  write-only; treats `idle && (finishReason === null || 'stop')` as the only
  designed skip (`null` covers Claude/Gemini, which never populate
  lastFinishReason, so their behavior is unchanged); and logs at error
  (instead of warn) when the batch was truncated or the output was malformed
  XML. This sits in the fall-through only — the newer overflow/quota/auth
  rejection branches above it (#3800) are untouched.
- OpenAICompatibleProvider.ts: threads finishReason through to
  session.lastFinishReason before each processAgentResponse call (init,
  observation, summary). The duplicate assistant-history push this fix
  originally also targeted was already removed upstream at #3619 (verified
  by reading OpenAICompatibleProvider.ts pre-change) — no source change
  needed for that part; tests/worker/openai-compatible-history.test.ts is
  added as a regression guard.

Tests: tsc clean; tests/worker tests/sdk tests/shared = 882 pass, 0 fail
(up from 838 pre-change, +44 new). Full `bun test`: 3053-3054 pass, 13 fail,
6 errors — identical failing-test set and tallies on a clean origin/main
checkout at the same commit (be44b6c), confirmed by running both full
suites side by side; none touch the files this change modifies. The 6
errors are workers/sync-hub's Cloudflare Workers test harness ("Cannot find
package 'cloudflare:test'"), unrelated and pre-existing.

Refs #3606, #3490 (max_tokens).

Co-authored-by: Gwen Ives <[email protected]>
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MwVTb8AbAAQZjxsrewXc83

* fix(openrouter): validate numeric settings as complete integers; scope finish reason to the response that produced it

Two review findings on the bounded-observer-context change (#3606):

- `parseOpenRouterInt` used `parseInt`, which accepts a numeric prefix, so a
  setting like `CLAUDE_MEM_OPENROUTER_ATTEMPT_TIMEOUT_MS: "1e3"` resolved to 1
  and every attempt timed out after one millisecond. It now accepts only a
  finite integral number or a full decimal-integer string (optional sign,
  surrounding whitespace trimmed) within [min, max]; anything else falls back
  to the documented default. `attemptTimeoutMs` additionally caps at the
  setTimeout ceiling (2^31-1 ms), which Node otherwise silently treats as
  ~1 ms.
- `session.lastFinishReason` was set by the OpenAI-compatible provider and
  never cleared, so a later empty Claude reply in the same session could read
  a stale `length` and be counted as a truncation instead of the designed
  skip. `processAgentResponse` now consumes and clears it at the top of every
  call, making the value response-scoped regardless of which branch handles
  the reply.

Tests: strict-integer cases (exponent, decimal, hex, sign, whitespace, empty,
non-numeric, non-ASCII digits, overflow, ceiling) and the stale-finish-reason
sequence (OpenRouter `length` followed by an empty Claude-path reply).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RJEKSU5eVwnvBf4irjtP4E

* fix(observer): anchor the schema in a system message, make the output cap configurable, name truncations

Trimmed to the three pieces main still lacked, rebuilt on main's request path
(buildOpenRouterRequestBody, the #4003 max_completion_tokens retry, the
trial-expiry fallback clearing and the gateway classification stay as #4276
left them):

- System anchor: a generation's framing prompt (marked by
  openObserverGeneration) goes out as the system message on OpenRouter and
  custom endpoints and as systemInstruction on Gemini. Its user-request block
  (or a fixed kickoff) stays the first user turn, so every request, the init
  one included, carries a user message and roles still alternate. Standalone
  requests (field condensation, the wrap-up) are unchanged.
- CLAUDE_MEM_OBSERVER_MAX_OUTPUT_TOKENS (default 4096, 256 to 1,000,000):
  max_tokens in buildOpenRouterRequestBody, max_completion_tokens on the
  compatibility retry, Gemini's maxOutputTokens, and the cap #3620's cut-off
  warning names. Validated on the settings API and documented.
- finishReason on ProviderQueryResult for OpenRouter and Gemini, a Gemini
  MAX_TOKENS warning for parity, and finishReason/truncated on the drop log
  line. The reason is scoped to one reply: processAgentResponse consumes and
  clears it. The invalid-output counter and the preserve policy stay as they
  are (C05).

Dropped as superseded: the sliding window and its message/char caps (main's
generation recycle and window-aware budget, #3625), the attempt timeout and
timeout:false (CLAUDE_MEM_LLM_TIMEOUT_MS), the per-provider
CLAUDE_MEM_OPENROUTER_MAX_TOKENS (one shared setting), the duplicate history
test (#3619) and the counter change.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01697ztU3scSYRfW49cPr9Du

---------

Co-authored-by: Gwen Ives <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>
Co-authored-by: Alex Newman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants