Skip to content

Commit 3e714a4

Browse files
anandgupta42claude
andcommitted
fix(learn): address the Codex review
- shared config schema accepts every `learn` key (including `enabled`), so the config API no longer drops them; SDK types and OpenAPI regenerated - file lessons are delivered for tools run inside the batch tool - a first message is never classified as a correction because of a later assistant reply - published playbooks load on teammates' checkouts; the exported skill is suppressed only where a same-name local approved store exists - a successful manual reflection clears the automatic backoff - export refuses unmanaged skill directories, unexpected files and symlinks - results doc: disclose two benchmark confounds found in review (topic-switch work directory name, compression-pair lesson order), tracked in #1408 Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
1 parent a7b80f9 commit 3e714a4

22 files changed

Lines changed: 630 additions & 84 deletions

File tree

‎docs/docs/configure/learn.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -389,7 +389,7 @@ Set these under `learn` in your project or user config. An environment variable
389389

390390
The numeric limits must be non-negative integers. A value of `0` turns that tier off. `learn status` prints the limits that are in effect.
391391

392-
The server config API (`/config`) accepts `capture`, `auto_reflect`, `model`, `core_lessons`, `retrieved_lessons`, `budget_tokens`, and `session_max_lessons`. Set the other keys in a config file or with their environment variables.
392+
The server config API (`/config`) accepts every `learn` key listed above. Config files, HTTP APIs, and generated SDK clients share the same schema.
393393

394394
## Cost
395395

‎packages/core/src/v1/config/config.ts‎

Lines changed: 36 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -244,31 +244,63 @@ export const Info = Schema.Struct({
244244
// altimate_change end
245245
}),
246246
),
247-
// altimate_change start - `learn` capture settings (local-only learning signals)
247+
// altimate_change start — share every learning setting across config files and HTTP APIs
248248
learn: Schema.optional(
249249
Schema.Struct({
250+
enabled: Schema.optional(Schema.Boolean).annotate({
251+
description:
252+
"Enable lesson delivery, capture, automatic reflection and the TUI reminder (default: true). Env: ALTIMATE_LEARN overrides in both directions.",
253+
}),
250254
capture: Schema.optional(Schema.Boolean).annotate({
251255
description:
252-
"Record learning signals (user corrections, repeated tool failures) to .altimate-code/learn/signals.jsonl (default: false). Local only, never uploaded. Env: ALTIMATE_LEARN_CAPTURE=1.",
256+
"Record learning signals (user corrections, repeated tool failures) to .altimate-code/learn/signals.jsonl (default: false). Stored locally; reflection sends redacted text to the model provider. Env: ALTIMATE_LEARN_CAPTURE=1.",
253257
}),
254258
auto_reflect: Schema.optional(Schema.Boolean).annotate({
255259
description:
256260
"At the end of `run`, reflect on open signals into the playbook candidate (default: false; requires capture). Env: ALTIMATE_LEARN_AUTO=1.",
257261
}),
262+
recovery_max_reflections: Schema.optional(NonNegativeInt).annotate({
263+
description:
264+
"Maximum reflections on leftover signals at startup (default: 3). Env: ALTIMATE_LEARN_RECOVERY_MAX_REFLECTIONS.",
265+
}),
266+
recovery_max_seconds: Schema.optional(NonNegativeInt).annotate({
267+
description:
268+
"Time budget in seconds for startup recovery (default: 300). Env: ALTIMATE_LEARN_RECOVERY_MAX_SECONDS.",
269+
}),
258270
model: Schema.optional(Schema.String).annotate({
259271
description: "Model (provider/model) for auto-reflect. Env: ALTIMATE_LEARN_MODEL. Default: the default model.",
260272
}),
273+
review_bots: Schema.optional(Schema.mutable(Schema.Array(Schema.String))).annotate({
274+
description: "Extra bot logins excluded by import-reviews (default: []).",
275+
}),
261276
core_lessons: Schema.optional(NonNegativeInt).annotate({
262277
description: "Maximum core lessons at session start (default: 15). Env: ALTIMATE_LEARN_CORE_LESSONS.",
263278
}),
264279
retrieved_lessons: Schema.optional(NonNegativeInt).annotate({
265280
description: "Maximum retrieved lessons at session start (default: 15). Env: ALTIMATE_LEARN_RETRIEVED_LESSONS.",
266281
}),
282+
request_lessons: Schema.optional(NonNegativeInt).annotate({
283+
description:
284+
"Maximum lessons added per request after session start (default: 5; 0 disables). Env: ALTIMATE_LEARN_REQUEST_LESSONS.",
285+
}),
286+
file_hook: Schema.optional(Schema.Boolean).annotate({
287+
description:
288+
"Deliver lessons matching file paths or identifiers (default: true). Env: ALTIMATE_LEARN_FILE_HOOK=0 disables.",
289+
}),
290+
file_lessons: Schema.optional(NonNegativeInt).annotate({
291+
description: "Maximum lessons added per file event (default: 5; 0 disables). Env: ALTIMATE_LEARN_FILE_LESSONS.",
292+
}),
267293
budget_tokens: Schema.optional(NonNegativeInt).annotate({
268-
description: "Token budget for the frozen Team rules section (default: 1500). Env: ALTIMATE_LEARN_BUDGET_TOKENS.",
294+
description:
295+
"Token budget for the frozen Team rules section (default: 1500). Env: ALTIMATE_LEARN_BUDGET_TOKENS.",
269296
}),
270297
session_max_lessons: Schema.optional(NonNegativeInt).annotate({
271-
description: "Maximum distinct lessons shown in one session (default: 40). Env: ALTIMATE_LEARN_SESSION_MAX_LESSONS.",
298+
description:
299+
"Maximum distinct lessons shown in one session (default: 40). Env: ALTIMATE_LEARN_SESSION_MAX_LESSONS.",
300+
}),
301+
max_stored: Schema.optional(PositiveInt).annotate({
302+
description:
303+
"Maximum stored lessons, enforced during curation; pinned lessons are retained (default: 1000). Env: ALTIMATE_LEARN_MAX_STORED.",
272304
}),
273305
}),
274306
),

‎packages/opencode/src/altimate/learn/capture.ts‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -105,22 +105,30 @@ export interface PartLike {
105105
}
106106

107107
export interface CaptureDeps {
108-
/** True when the session has a completed assistant message other than the one being classified. */
109-
hasPriorAssistant: (sessionID: string, exceptMessageID: string) => Promise<boolean>
108+
/** True when the session has a completed assistant message before the user message being classified. */
109+
hasPriorAssistant: (sessionID: string, beforeMessageID: string) => Promise<boolean>
110110
record: (signal: NewSignal) => Promise<Signal | undefined>
111111
}
112112

113113
export class Capture {
114114
private userMessages = new Set<string>()
115-
private sessionsWithAssistant = new Set<string>()
115+
private sessionsWithAssistant = new Map<string, string>()
116116
private trackers = new Map<string, ToolRetryTracker>()
117117
private pending = new Set<Promise<unknown>>()
118118

119119
constructor(private deps: CaptureDeps) {}
120120

121121
onMessage(info: MessageLike) {
122122
if (info.role === "user") remember(this.userMessages, info.id, RECENT_ID_CAP)
123-
else if (info.role === "assistant" && info.time?.completed) remember(this.sessionsWithAssistant, info.sessionID, SESSION_CAP)
123+
else if (info.role === "assistant" && info.time?.completed) this.rememberAssistant(info.sessionID, info.id)
124+
}
125+
126+
private rememberAssistant(sessionID: string, messageID: string) {
127+
const earliest = this.sessionsWithAssistant.get(sessionID)
128+
this.sessionsWithAssistant.delete(sessionID)
129+
this.sessionsWithAssistant.set(sessionID, earliest && earliest < messageID ? earliest : messageID)
130+
if (this.sessionsWithAssistant.size > SESSION_CAP)
131+
this.sessionsWithAssistant.delete(this.sessionsWithAssistant.keys().next().value!)
124132
}
125133

126134
onPart(part: PartLike) {
@@ -144,9 +152,11 @@ export class Capture {
144152
if (!this.userMessages.has(part.messageID) || part.synthetic || part.ignored || !part.text) return
145153
const reason = correctionReason(part.text)
146154
if (!reason) return
147-
const known = this.sessionsWithAssistant.has(part.sessionID)
155+
const prior = this.sessionsWithAssistant.get(part.sessionID)
156+
const known = prior !== undefined && prior < part.messageID
148157
if (!known && !(await this.deps.hasPriorAssistant(part.sessionID, part.messageID))) return
149-
remember(this.sessionsWithAssistant, part.sessionID, SESSION_CAP)
158+
// A successful history lookup proves an assistant exists before this ID, not before older messages.
159+
this.rememberAssistant(part.sessionID, part.messageID)
150160
await this.deps.record({
151161
kind: "user_correction",
152162
sessionID: part.sessionID,
@@ -213,9 +223,9 @@ export async function startCapture(ctx: { directory: string; worktree: string })
213223
const { SessionID } = await import("../../session/schema")
214224
const capture = new Capture({
215225
record: (signal) => appendSignal(root, signal),
216-
hasPriorAssistant: async (sessionID, exceptMessageID) => {
226+
hasPriorAssistant: async (sessionID, beforeMessageID) => {
217227
const messages = await Session.messages({ sessionID: SessionID.make(sessionID) })
218-
return messages.some((m) => m.info.role === "assistant" && m.info.id !== exceptMessageID && !!m.info.time.completed)
228+
return messages.some((m) => m.info.role === "assistant" && m.info.id < beforeMessageID && !!m.info.time.completed)
219229
},
220230
})
221231
active.add(capture)

‎packages/opencode/src/altimate/learn/delivery.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -345,6 +345,23 @@ export class Delivery {
345345
})
346346
}
347347

348+
/** Share executed-path selection between direct calls and batch inner calls. */
349+
async appendFileLessons(session: string, tool: string, args: Record<string, unknown>, result: { output: string; metadata?: unknown }): Promise<void> {
350+
if (!["read", "edit", "write", "patch", "apply_patch"].includes(tool)) return
351+
try {
352+
const paths: string[] = []
353+
if (typeof args.filePath === "string") paths.push(args.filePath)
354+
const changed = (result.metadata as { files?: { filePath: string; movePath?: string }[] } | undefined)?.files
355+
for (const file of changed ?? []) paths.push(file.filePath, ...(file.movePath ? [file.movePath] : []))
356+
for (const file of new Set(paths)) {
357+
const note = await this.file(session, file)
358+
if (note) result.output += `\n\n${note}`
359+
}
360+
} catch (error) {
361+
log.warn("learn file selection failed", { error })
362+
}
363+
}
364+
348365
private async finishFlush() {
349366
const file = path.join(this.dir, ".flush.json")
350367
const raw = await read(file)

‎packages/opencode/src/altimate/learn/store.ts‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -258,11 +258,27 @@ export async function saveCandidate(root: string, name: string, pb: Playbook.Pla
258258
})
259259
}
260260

261-
/** Publish only on demand. Managed exports remain excluded from local auto-loading. */
261+
/** Publish only on demand. Approved local stores replace their exports in auto-loading. */
262262
export async function exportSkill(root: string, name: string): Promise<string> {
263263
return transaction(root, async () => {
264-
const lessons = await loadApproved(root, name)
265264
const p = paths(root, name)
265+
// Check before loading lessons: legacy migration can also rename malformed SKILL.md files.
266+
const target = await fs.lstat(p.skillDir).catch((error: NodeJS.ErrnoException) => {
267+
if (error.code !== "ENOENT") throw error
268+
})
269+
if (target) {
270+
const refuse = (reason: string) => new StoreError(
271+
`Refusing to export "${name}": "${p.skillDir}" must be a learn-managed single-file export (${reason}).`,
272+
)
273+
if (!target.isDirectory() || target.isSymbolicLink()) throw refuse("target is not a regular directory; symlinks are not allowed")
274+
const entries = await fs.readdir(p.skillDir, { withFileTypes: true })
275+
if (entries.length !== 1 || entries[0].name !== "SKILL.md" || !entries[0].isFile())
276+
throw refuse("expected only a regular SKILL.md file, with no extra files, directories, or symlinks")
277+
const existing = await fs.readFile(p.skill, "utf8")
278+
if (!existing.split(/\r?\n/).includes(Playbook.HEADER) || validateLegacy(name, existing))
279+
throw refuse("SKILL.md is not an unchanged learn-managed export")
280+
}
281+
const lessons = await loadApproved(root, name)
266282
if (await read(p.approved) === undefined) throw new StoreError(`No approved lessons for "${name}".`)
267283
const applyPaths = lessons.length && lessons.every((lesson) => lesson.trigger?.paths?.length)
268284
? [...new Set(lessons.flatMap((lesson) => lesson.trigger!.paths!))]

‎packages/opencode/src/cli/cmd/learn.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ import { errText, prepareReflection, reflectCore, reflectSessionSignals, sourceF
3434
import { bootstrap, DEFAULT_BOOTSTRAP_LIMIT, DEFAULT_MAX_REFLECTIONS, DEFAULT_MAX_SECONDS, type BootstrapModel } from "../../altimate/learn/bootstrap"
3535
import { importReviews } from "../../altimate/learn/import-reviews"
3636
import { formatUsage } from "../../altimate/learn/usage"
37+
// altimate_change start — clear automatic retry backoff after committed manual reflection
38+
import { recordReflection } from "../../altimate/learn/schedule-state"
39+
// altimate_change end
3740
// altimate_change start — log best-effort learning maintenance failures
3841
import { Log } from "../../util/log"
3942

@@ -575,6 +578,10 @@ const ReflectCommand = effectCmd({
575578
}
576579
if (attempt.r.status === "none") continue
577580
const { result, signals, kind } = attempt.r
581+
// altimate_change start — committed manual success clears backoff for this named store
582+
yield* Effect.promise(() => recordReflection(root, sessionID, "success", summarize(result.curated), name, undefined, result.usage)
583+
.catch((error) => log.warn("manual reflection status update failed", { error: redactSecrets(errText(error)) })))
584+
// altimate_change end
578585
if (!args.json)
579586
out(`Session ${sessionID}: learned from ${signals.length} signal${signals.length === 1 ? "" : "s"} (feedback kind: ${kind}).`)
580587
const json = report(result, { session: sessionID, signals: signals.length, feedbackKind: kind })

‎packages/opencode/src/config/config.ts‎

Lines changed: 3 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -219,44 +219,9 @@ async function resolveLoadedPlugins<T extends { plugin?: ConfigPluginV1.Spec[] }
219219
return config
220220
}
221221

222-
// altimate_change start — local learning settings without changing the shared SDK schema
223-
const LocalInfo = Schema.Struct({
224-
...ConfigV1.Info.fields,
225-
learn: Schema.optional(Schema.Struct({
226-
enabled: Schema.optional(Schema.Boolean).annotate({
227-
description: "Enable lesson delivery, capture, automatic reflection and the TUI reminder (default: true). Env: ALTIMATE_LEARN overrides in both directions.",
228-
}),
229-
capture: Schema.optional(Schema.Boolean),
230-
auto_reflect: Schema.optional(Schema.Boolean),
231-
recovery_max_reflections: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))),
232-
recovery_max_seconds: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))),
233-
model: Schema.optional(Schema.String),
234-
review_bots: Schema.optional(Schema.Array(Schema.String)),
235-
core_lessons: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))),
236-
retrieved_lessons: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))),
237-
request_lessons: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))).annotate({
238-
description: "Maximum lessons added per request after session start (default: 5; 0 disables). Env: ALTIMATE_LEARN_REQUEST_LESSONS.",
239-
}),
240-
file_hook: Schema.optional(Schema.Boolean).annotate({
241-
description: "Deliver lessons matching file paths or identifiers (default: true). Env: ALTIMATE_LEARN_FILE_HOOK=0 disables.",
242-
}),
243-
file_lessons: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))).annotate({
244-
description: "Maximum lessons added per file event (default: 5; 0 disables). Env: ALTIMATE_LEARN_FILE_LESSONS.",
245-
}),
246-
budget_tokens: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))),
247-
session_max_lessons: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(0))),
248-
max_stored: Schema.optional(Schema.Number.check(Schema.isInt(), Schema.isGreaterThanOrEqualTo(1))).annotate({
249-
description: "Maximum stored lessons, enforced during curation; pinned lessons are retained (default: 1000). Env: ALTIMATE_LEARN_MAX_STORED.",
250-
}),
251-
})),
252-
})
253-
type LocalInfo = ConfigV1.Info & {
254-
learn?: ConfigV1.Info["learn"] & {
255-
enabled?: boolean
256-
request_lessons?: number; file_hook?: boolean; file_lessons?: number
257-
max_stored?: number; recovery_max_reflections?: number; recovery_max_seconds?: number; review_bots?: readonly string[]
258-
}
259-
}
222+
// altimate_change start — keep local config validation consistent with the HTTP and SDK schema
223+
const LocalInfo = ConfigV1.Info
224+
type LocalInfo = ConfigV1.Info
260225
// altimate_change end
261226

262227
// altimate_change start — opencode config includes local learn settings

‎packages/opencode/src/session/prompt.ts‎

Lines changed: 4 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -2234,7 +2234,9 @@ export namespace SessionPrompt {
22342234
abort: options.abortSignal!,
22352235
messageID: input.processor.message.id,
22362236
callID: options.toolCallId,
2237-
extra: { model: input.model, bypassAgentCheck: input.bypassAgentCheck },
2237+
// altimate_change start — batch inner calls reuse the prompt's lesson delivery state
2238+
extra: { model: input.model, bypassAgentCheck: input.bypassAgentCheck, lessons: input.lessons },
2239+
// altimate_change end
22382240
agent: input.agent.name,
22392241
// altimate_change start — fork MessageV2.WithParts ≡ core SessionV1.WithParts at the Tool.Context boundary
22402242
messages: input.messages as unknown as Tool.Context["messages"],
@@ -2350,20 +2352,7 @@ export namespace SessionPrompt {
23502352
// altimate_change end
23512353
)
23522354
// altimate_change start — approved file rules use executed paths, never file contents.
2353-
if (input.lessons && ["read", "edit", "write", "patch", "apply_patch"].includes(item.id)) {
2354-
try {
2355-
const paths: string[] = []
2356-
if (typeof args.filePath === "string") paths.push(args.filePath)
2357-
const changed = (result.metadata as { files?: { filePath: string; movePath?: string }[] } | undefined)?.files
2358-
for (const file of changed ?? []) paths.push(file.filePath, ...(file.movePath ? [file.movePath] : []))
2359-
for (const file of new Set(paths)) {
2360-
const note = await input.lessons.file(ctx.sessionID, file)
2361-
if (note) stamped.output += `\n\n${note}`
2362-
}
2363-
} catch (error) {
2364-
log.warn("learn file selection failed", { error })
2365-
}
2366-
}
2355+
await input.lessons?.appendFileLessons(ctx.sessionID, item.id, args, stamped)
23672356
// altimate_change end
23682357
// altimate_change start — return the source-stamped output
23692358
return stamped

‎packages/opencode/src/session/system.ts‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import { Instance } from "../project/instance"
55
import { Glob } from "../util/glob"
66
import { Log } from "../util/log"
77
import { HEADER as LEARN_MANAGED_HEADER } from "../altimate/learn/playbook"
8+
import { paths as learnPaths } from "../altimate/learn/store"
89
// altimate_change end
910

1011
import PROMPT_ANTHROPIC from "./prompt/anthropic.txt"
@@ -218,9 +219,16 @@ export namespace SystemPrompt {
218219
async function collectAutoLoadedSkills(list: Skill.Info[]): Promise<Skill.Info[]> {
219220
const out: Skill.Info[] = []
220221
for (const skill of list) {
221-
// altimate_change start — learned rules live in the lesson store; the retained
222-
// legacy skill and explicit publish exports must not inject a stale second copy.
223-
if (skill.content.includes(LEARN_MANAGED_HEADER)) continue
222+
// altimate_change start — suppress a stale export only when this checkout has
223+
// approved local lessons; teammates receive the published skill without that store.
224+
if (skill.content.includes(LEARN_MANAGED_HEADER)) {
225+
const root = Instance.worktree !== "/" ? Instance.worktree : Instance.directory
226+
try {
227+
if (await Bun.file(learnPaths(root, skill.name).approved).exists()) continue
228+
} catch (err) {
229+
autoLoadLog.warn("local approved lesson store unavailable", { skill: skill.name, err })
230+
}
231+
}
224232
// altimate_change end
225233
if (skill.alwaysApply === true) {
226234
out.push(skill)

‎packages/opencode/src/tool/batch.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,9 @@ import { ProviderID, ModelID } from "../provider/schema"
66
import { AppRuntime } from "../effect/app-runtime"
77
import type { LegacyContext } from "../altimate/tool-zod-compat"
88
// altimate_change end
9+
// altimate_change start — batch inner calls share the prompt's lesson delivery state
10+
import type { Delivery as LessonDelivery } from "../altimate/learn/delivery"
11+
// altimate_change end
912
import DESCRIPTION from "./batch.txt"
1013

1114
const DISALLOWED = new Set(["batch"])
@@ -103,6 +106,10 @@ export const BatchTool = Tool.define("batch", {
103106
// altimate_change start — v1.17.9: Tool.Def.execute returns an Effect; run via AppRuntime
104107
const result = await AppRuntime.runPromise(tool.execute(validatedParams, toEffectContext(ctx, partID)))
105108
// altimate_change end
109+
// altimate_change start — append file lessons before persisting each inner result
110+
const lessons = ctx.extra?.lessons as LessonDelivery | undefined
111+
await lessons?.appendFileLessons(ctx.sessionID, call.tool, call.parameters, result)
112+
// altimate_change end
106113
const attachments = result.attachments?.map((attachment) => ({
107114
...attachment,
108115
id: PartID.ascending(),

0 commit comments

Comments
 (0)