Skip to content

Commit cd539a8

Browse files
anandgupta42claude
andcommitted
fix(learn): compaction-safe turn starts and exposure-scoped ambiguity
- A request right after a compaction marker may be its replay, so its turn start is unknown; compaction markers are never request boundaries. - Extract `turnMessages` (synthetic and compaction flags from real parts) and test it. - Ambiguity counts only lessons the session was shown in their current text; a later lesson created from the correction no longer hides recurrence. - Mark a correction message handled only after its signal is recorded, so a failed write is retried by a later update. - Document recurrence in `learn status` and the reflection `r:N` hint. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 1a22075 commit cd539a8

5 files changed

Lines changed: 92 additions & 17 deletions

File tree

‎docs/docs/configure/learn.md‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -222,7 +222,21 @@ altimate-code learn disable
222222

223223
### status
224224

225-
Shows whether learning is on, whether automatic promotion is on with its last promotion and last held-back reason, lesson counts (approved, candidate, retired), open signals, pending recoveries, the last reflection and its summary, and the resolved limits. `--json` also includes the last reflection's token use and estimated cost.
225+
Shows whether learning is on, whether automatic promotion is on with its last promotion and last held-back reason, lesson counts (approved, candidate, retired), open signals, pending recoveries, the last reflection and its summary, the resolved limits, and lesson recurrence. `--json` also includes the last reflection's token use and estimated cost.
226+
227+
**Recurrence** answers "was this lesson shown, and did the user still correct the same thing?". It reads the newest 200 session snapshots and the recorded corrections. It makes no model calls and writes nothing. For each shown lesson it counts sessions where a matching correction came:
228+
229+
| Count | Meaning |
230+
|---|---|
231+
| `recurred` | after the lesson was shown, at the start of the corrected request or earlier |
232+
| `during` | while the agent was answering, for example a file-hook lesson added mid-turn; the lesson may have arrived too late |
233+
| `unknown` | in an order that cannot be told, for example older corrections or a request replayed after compaction |
234+
| `ambiguous` | when the correction matched more than one lesson the session was shown |
235+
| `earlier_version` | to a different version of the lesson text; it never counts for the current text |
236+
237+
A correction matches a lesson when it names one of the lesson's backtick identifiers plus another word from the lesson, or contains most of the lesson's words. The text output lists only lessons with a matching correction and summarizes the rest as "no observed correction". That is not proof the agent followed them. Corrections are recorded only while capture is on. Status also lists up to 5 identifiers that were corrected in 2 or more sessions with no matching current lesson. They are taken from your local corrections and never leave the project. If the analysis fails, status shows `Recurrence unavailable` (`recurrence_error` in JSON), and the rest of the status still prints.
238+
239+
When the current text of a lesson has `recurred` in 2 or more sessions, reflection marks it `r:N` in the playbook it sends the model. The model is asked to make the lesson clearer or more specific, never weaker. The result is a candidate like any other and goes through the same review or automatic-promotion gates.
226240

227241
```bash
228242
altimate-code learn status

‎packages/opencode/src/altimate/learn/capture.ts‎

Lines changed: 25 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -92,12 +92,30 @@ export interface MessageLike {
9292
time?: { created?: number; completed?: number }
9393
}
9494

95-
/** Start of the agent's response to the request the correction answers; synthetic continuations are not requests. */
96-
export function correctedTurnStart(messages: readonly (MessageLike & { synthetic?: boolean })[], messageID: string): number | undefined {
95+
export type TurnMessage = MessageLike & { synthetic?: boolean; compaction?: boolean }
96+
97+
/** Session history as turn boundaries: all-synthetic text continues a turn; a compaction part marks compaction. */
98+
export function turnMessages(messages: readonly { info: MessageLike; parts: readonly { type: string; synthetic?: boolean }[] }[]): TurnMessage[] {
99+
return messages.map(({ info, parts }) => {
100+
const text = parts.filter((part) => part.type === "text")
101+
return {
102+
...info,
103+
synthetic: text.length > 0 && text.every((part) => part.synthetic),
104+
compaction: parts.some((part) => part.type === "compaction"),
105+
}
106+
})
107+
}
108+
109+
/**
110+
* Start of the agent's response to the request the correction answers. Synthetic continuations are not
111+
* requests. A request right after a compaction may be its replay, so its turn start is unknown.
112+
*/
113+
export function correctedTurnStart(messages: readonly TurnMessage[], messageID: string): number | undefined {
97114
const end = messages.findIndex((message) => message.id === messageID && message.role === "user")
98115
if (end < 0) return
99-
const previous = messages.slice(0, end).findLastIndex((message) => message.role === "user" && !message.synthetic)
116+
const previous = messages.slice(0, end).findLastIndex((message) => message.role === "user" && !message.synthetic && !message.compaction)
100117
if (previous < 0) return
118+
if (messages.slice(0, previous).findLast((message) => message.role === "user")?.compaction) return
101119
return messages.slice(previous + 1, end).find((message) => message.role === "assistant")?.time?.created
102120
}
103121

@@ -181,12 +199,11 @@ export class Capture {
181199
if (message.role !== "user") return
182200
const reason = correctionReason(part.text)
183201
if (!reason) return
202+
// Later updates of a recorded correction deduplicate to its first signal; skip their history reads.
203+
if (this.corrected.has(part.messageID)) return
184204
const prior = this.sessionsWithAssistant.get(part.sessionID)
185205
const known = prior !== undefined && message.time?.created !== undefined && prior < message.time.created
186206
if (!known && !(await this.deps.hasPriorAssistant(part.sessionID, part.messageID))) return
187-
// Later updates of the same message deduplicate to the first signal; skip their history reads.
188-
if (this.corrected.has(part.messageID)) return
189-
remember(this.corrected, part.messageID, RECENT_ID_CAP)
190207
const start = await this.deps.turnStart?.(part.sessionID, part.messageID).catch((error) => {
191208
log.warn("learn turn start unavailable", { error })
192209
return undefined
@@ -199,6 +216,7 @@ export class Capture {
199216
text: part.text,
200217
reason,
201218
})
219+
remember(this.corrected, part.messageID, RECENT_ID_CAP)
202220
}
203221

204222
private async toolPart(part: PartLike) {
@@ -260,10 +278,7 @@ export async function startCapture(ctx: { directory: string; worktree: string })
260278
record: (signal) => appendSignal(root, signal),
261279
turnStart: async (sessionID, messageID) => {
262280
const messages = await Session.messages({ sessionID: SessionID.make(sessionID) })
263-
return correctedTurnStart(messages.map((message) => {
264-
const text = message.parts.filter((part) => part.type === "text")
265-
return { ...message.info, synthetic: text.length > 0 && text.every((part) => part.synthetic) }
266-
}), messageID)
281+
return correctedTurnStart(turnMessages(messages), messageID)
267282
},
268283
hasPriorAssistant: async (sessionID, beforeMessageID) => {
269284
const messages = await Session.messages({ sessionID: SessionID.make(sessionID) })

‎packages/opencode/src/altimate/learn/recurrence.ts‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -73,13 +73,15 @@ export function attribute(signals: readonly Signal[], snapshots: readonly ShownS
7373
}
7474
continue
7575
}
76-
for (const lesson of matches) {
76+
// Only lessons this session saw in their current text compete; one never shown cannot explain the correction.
77+
const exposed = matches.flatMap((lesson) => {
7778
const counts = rows.get(key(lesson.name, lesson.id, lesson.textHash))?.counts
78-
if (!counts) continue
7979
const shown = sessions.get(signal.sessionID)!.filter((entry) =>
8080
entry.name === lesson.name && entry.id === lesson.id && entry.textHash === lesson.textHash)
81-
if (!shown.length) continue
82-
if (matches.length > 1) {
81+
return counts && shown.length ? [{ counts, shown }] : []
82+
})
83+
for (const { counts, shown } of exposed) {
84+
if (exposed.length > 1) {
8385
counts.ambiguous.add(signal.sessionID)
8486
continue
8587
}

‎packages/opencode/test/altimate/learn/capture.test.ts‎

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ import { afterEach, beforeEach, describe, expect, test } from "bun:test"
33
import fs from "node:fs/promises"
44
import os from "node:os"
55
import path from "node:path"
6-
import { correctedTurnStart, Capture, RETRY_THRESHOLD, ToolRetryTracker, type PartLike } from "../../../src/altimate/learn/capture"
6+
import { correctedTurnStart, turnMessages, Capture, RETRY_THRESHOLD, ToolRetryTracker, type PartLike } from "../../../src/altimate/learn/capture"
77
import * as Signals from "../../../src/altimate/learn/signals"
88

99
const call = (id: string, tool: string, status: "error" | "completed" | "running", error = "boom", messageID = "m1") => ({
@@ -358,6 +358,43 @@ describe("correction turn start", () => {
358358
expect(correctedTurnStart(continued, "correction")).toBe(2000)
359359
expect(correctedTurnStart(continued.map((m) => ({ ...m, synthetic: false })), "correction")).toBe(4000)
360360
})
361+
test("history parts mark synthetic continuations and compaction markers", () => {
362+
const info = (id: string, role: string, created: number) => ({ id, sessionID: "s", role, time: { created, completed: created + 1 } })
363+
const mapped = turnMessages([
364+
{ info: info("u1", "user", 1000), parts: [{ type: "text" }, { type: "file" }] },
365+
{ info: info("a1", "assistant", 2000), parts: [{ type: "text" }] },
366+
{ info: info("continue", "user", 3000), parts: [{ type: "text", synthetic: true }] },
367+
{ info: info("marker", "user", 3500), parts: [{ type: "compaction" }] },
368+
{ info: info("files", "user", 3600), parts: [{ type: "file" }] },
369+
])
370+
expect(mapped.map(({ synthetic, compaction }) => [synthetic, compaction])).toEqual([
371+
[false, false], [false, false], [true, false], [false, true], [false, false],
372+
])
373+
})
374+
test("a replayed request after compaction has no known turn start", () => {
375+
const at = (id: string, role: string, created: number, extra = {}) => ({ id, sessionID: "s", role, time: { created, completed: created + 1 }, ...extra })
376+
const compacted = [
377+
at("u1", "user", 1000), at("step", "assistant", 2000),
378+
at("marker", "user", 3000, { compaction: true }), at("summary", "assistant", 3100),
379+
at("replay", "user", 3200), at("final", "assistant", 4000), at("correction", "user", 5000),
380+
]
381+
expect(correctedTurnStart(compacted, "correction")).toBeUndefined()
382+
// A manual compaction without a replay leaves the original request as the boundary.
383+
expect(correctedTurnStart(compacted.filter((m) => m.id !== "replay"), "correction")).toBe(2000)
384+
})
385+
test("a failed record is retried by a later update", async () => {
386+
let attempts = 0
387+
const capture = new Capture({
388+
hasPriorAssistant: async () => true,
389+
turnStart: async (_, id) => correctedTurnStart(history, id),
390+
record: async () => { if (attempts++ === 0) throw new Error("lock timeout"); return undefined },
391+
})
392+
for (const info of history) capture.onMessage(info)
393+
for (const text of ["No, use UTC", "No, use UTC instead."])
394+
await capture.onPart({ id: "part", messageID: "correction", sessionID: "s", type: "text", text })
395+
await capture.flush()
396+
expect(attempts).toBe(2)
397+
})
361398
test("repeated updates of one correction look up history once", async () => {
362399
let lookups = 0
363400
const capture = new Capture({

‎packages/opencode/test/altimate/learn/recurrence.test.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,14 @@ describe("recurrence attribution", () => {
9999
shown.shown.push({ ...snapshot().shown[0], name: other.name })
100100
const result = run([signal()], [shown], [lesson, other])
101101
expect(result.lessons.find((row) => row.name === lesson.name)).toMatchObject({ ambiguous: 0, earlier_version: 1 })
102-
expect(result.lessons.find((row) => row.name === other.name)).toMatchObject({ ambiguous: 1 })
102+
expect(result.lessons.find((row) => row.name === other.name)).toMatchObject({ ambiguous: 0, recurred: 1 })
103+
})
104+
105+
test("a matching lesson the session never saw does not hide recurrence", () => {
106+
const added: NamedLesson = { name: "team-playbook", lesson: { id: "L-0003", text: "Always convert `event_at` timestamps with `to_utc` first." } }
107+
const result = run([signal("s1"), signal("s2")], [snapshot("s1"), snapshot("s2")], [lesson, added])
108+
expect(result.lessons.find((row) => row.id === lesson.lesson.id)).toMatchObject({ recurred: 2, ambiguous: 0 })
109+
expect(recurrenceHint(result, lesson.name, lesson.lesson)).toBe(2)
103110
})
104111

105112
test("earlier lesson version cannot supply current evidence", () => {

0 commit comments

Comments
 (0)