Skip to content

Commit 2803138

Browse files
anandgupta42claude
andcommitted
fix(learn): no helpful vote for a correct lesson the agent missed
The reflect prompt told the model that feedback asking for exactly what a lesson says is HELPFUL, even when the agent had ignored that lesson. An ignored lesson gained helpful votes, which raise its core ranking and offset harmful votes, and it contradicted the new `r:N` hint. HELPFUL now requires the digest to show the agent followed the lesson; a correct but missed lesson keeps its counters and is edited only if its wording is unclear. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 86cc88f commit 2803138

2 files changed

Lines changed: 9 additions & 2 deletions

File tree

‎packages/opencode/src/altimate/learn/prompt.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@ How to work:
1616
2. Infer general conventions. State one rule with the exact identifier it governs, without rationale or incident details. Keep the specific table, model, column, or file identifier when the correction depends on it; do not substitute a vague generality.
1717
3. Learn only from the external feedback. Do not treat the agent's own claims of success as evidence.
1818
4. For an existing bullet:
19-
- HELPFUL when the feedback shows the agent followed it and the related checks passed.
20-
- HARMFUL only with explicit evidence that the agent FOLLOWED the bullet and that doing so caused a failure or a reviewer objection. The agent not following a bullet, feedback that is unrelated to the bullet, or feedback that asks for exactly what the bullet says is never HARMFUL (the last is HELPFUL). If unsure, leave the bullet alone.
19+
- HELPFUL only when the DIGEST shows the agent followed it and the feedback shows the related checks passed. When the feedback asks for exactly what the bullet says because the agent did not follow it, the bullet is right but was missed: it is neither HELPFUL nor HARMFUL. Leave its counters alone, and EDIT it only if its wording is unclear.
20+
- HARMFUL only with explicit evidence that the agent FOLLOWED the bullet and that doing so caused a failure or a reviewer objection. The agent not following a bullet, feedback that is unrelated to the bullet, or feedback that asks for exactly what the bullet says is never HARMFUL. If unsure, leave the bullet alone.
2121
- When feedback states a different expected behaviour than a bullet, EDIT the bullet to that expected behaviour (or ADD with "supersedes"). HARMFUL alone loses the knowledge. Keep the corrected convention general.
2222
- REMOVE when the feedback shows it is plainly wrong or outdated.
2323
5. Before ADD or EDIT, compare its code identifiers, naming prefixes and suffixes in backticks with every existing bullet. If another bullet touches the same convention, explicitly resolve their relationship: EDIT the existing rule, REMOVE the outdated rule, ADD a replacement with "supersedes": "<existing id>", or ADD a compatible narrower rule with "coexists": ["<existing id>", ...]. For example, a rule that analyses keep integer `*_cents` columns can coexist with a rule that staging models convert `*_cents` columns to dollars. EDIT also needs "coexists" for every other overlapping bullet it remains compatible with. Refer only to existing bullet ids. Two bullets that disagree must never both remain. When feedback contradicts a bullet, that bullet is outdated: supersede or remove it; do not merely ADD a conflicting rule or mark the old one HARMFUL.

‎packages/opencode/test/altimate/learn/reflect.test.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,13 @@ describe("buildPrompt", () => {
1919
}
2020
})
2121

22+
test("a missed but correct bullet earns no helpful vote", () => {
23+
const { system } = buildPrompt({ digest: "d", feedback: "f", kind: "user", bullets })
24+
expect(system).toContain("HELPFUL only when the DIGEST shows the agent followed it")
25+
expect(system).toContain("it is neither HELPFUL nor HARMFUL")
26+
expect(system).not.toContain("(the last is HELPFUL)")
27+
})
28+
2229
test("requires backticks around generated identifiers and naming patterns", () => {
2330
const { system } = buildPrompt({ digest: "d", feedback: "f", kind: "ci", bullets })
2431
expect(system).toContain("Wrap every code identifier and naming pattern (including prefixes and suffixes) in backticks.")

0 commit comments

Comments
 (0)