Skip to content

Commit 48f4655

Browse files
committed
fix(core): bound session title generation with a small output limit
The background title request went out with no max_tokens, so on a local thinking model it reasoned in an unbounded loop for minutes while competing with the main request for CPU (fixes anomalyco#54401). Titles now get a 512-token ceiling through the same outputLimit path as primary and compaction requests; hooks can still change or remove the limit.
1 parent ffe6fed commit 48f4655

2 files changed

Lines changed: 26 additions & 3 deletions

File tree

‎packages/core/src/session/model-request.ts‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,9 @@ const GENERATION_KEYS = new Set(Object.keys(GenerationOptions.fields))
5555
const OUTPUT_TOKEN_FALLBACK = 32_000
5656
// No reply needs more, however much the model allows.
5757
const OUTPUT_TOKEN_MAX = 256_000
58+
// A title never needs more; without a bound a local thinking model can reason in circles
59+
// for minutes on the background title request while the user waits (#54401).
60+
const TITLE_OUTPUT_MAX = 512
5861
// Prompt text is estimated at about 4 characters per token, which can run low on dense text such as code.
5962
const ESTIMATE_ERROR = 0.15
6063
// Share of the window a turn leaves unused when its output limit is fitted to the room left. Providers count a few
@@ -281,8 +284,9 @@ export const layer = Layer.effect(
281284
(t) => [{ description: t.description, input: JSON.parse(JSON.stringify(t.inputSchema)) }, t] as const,
282285
),
283286
)
284-
// Hooks see the default output limit and may change or remove it. Titles and generate keep the provider default,
285-
// because their reasoning is hard to budget.
287+
// Hooks see the default output limit and may change or remove it. Titles get a small
288+
// bound so background thinking-model runs cannot outpace the main request; generate
289+
// keeps the provider default, because its reasoning is hard to budget.
286290
const shaped = yield* shape(
287291
{
288292
sessionID: session.id,
@@ -294,7 +298,9 @@ export const layer = Layer.effect(
294298
? { maxTokens: outputLimit(model.limit, input.inputTokens, WINDOW_HEADROOM) }
295299
: kind === "compaction"
296300
? { maxTokens: outputLimit(model.limit, input.inputTokens) }
297-
: {},
301+
: kind === "title"
302+
? { maxTokens: Math.min(outputLimit(model.limit, input.inputTokens), TITLE_OUTPUT_MAX) }
303+
: {},
298304
},
299305
Object.fromEntries(Array.from(given, ([d, t]) => [t.name, d])),
300306
)

‎packages/core/test/session-title.test.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -676,3 +676,20 @@ it.effect("preserves a manual rename completed while generation is in flight", (
676676
expect((yield* store.get(sessionID))?.title).toBe("Manual title")
677677
}),
678678
)
679+
680+
it.effect("bounds the title request with a small output limit (#54401)", () =>
681+
Effect.gen(function* () {
682+
yield* enableTitleAgent
683+
const sessionID = Session.ID.make("ses_title_limit")
684+
yield* insertSession(sessionID)
685+
yield* prompt(sessionID, "Hello")
686+
687+
const title = yield* SessionTitle.Service
688+
yield* title.generate(sessionID)
689+
690+
expect(requests).toHaveLength(1)
691+
const maxTokens = requests[0]?.generation?.maxTokens
692+
expect(maxTokens).toBeDefined()
693+
expect(maxTokens!).toBeLessThanOrEqual(512)
694+
}),
695+
)

0 commit comments

Comments
 (0)