Skip to content

Commit 133d4dd

Browse files
committed
docs(triage): record the stack outcome and what the audit gates caught
Five layers, sixteen bug issues dispositioned, zero closed on a partial fix. Twelve adversarial rounds, nine FAIL — including three #915 defects that were each introduced by fixing the previous one, which is the part worth remembering: a hand-written allowlist was never the right answer where the domain is an open string.
1 parent 151da16 commit 133d4dd

1 file changed

Lines changed: 106 additions & 0 deletions

File tree

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# 070 — Outcome
2+
3+
## What shipped
4+
5+
A four-layer stack off `origin/dev` at `14b20def2`, each layer targeting the
6+
one before it.
7+
8+
| PR | Layer | Contents |
9+
|---|---|---|
10+
| #951 | 1/5 | triage: 22 label corrections + this plan unit |
11+
| #952 | 2/5 | #908 long-context pricing tiers |
12+
| #953 | 3/5 | six carried contributor bug fixes |
13+
| #954 | 4/5 | #545 classifier thinking round-trip |
14+
| #955 | 5/5 | #915 cooldown early-recovery probe |
15+
16+
All four checks green on every layer. `enforce-target` passes on the stacked
17+
children, which is the gate that decides whether this shape is allowed at all.
18+
19+
## Disposition of the sixteen bug issues
20+
21+
| Outcome | Issues |
22+
|---|---|
23+
| fixed in this stack | #908, #545, #915 |
24+
| blocked upstream, evidenced | #907 (canonical source in `lidge-jun/jawcode`) |
25+
| stale evidence, re-test requested | #875 (tested 115 commits before the fix) |
26+
| reclassified | #919 → `enhancement` (deliberate policy, not a defect) |
27+
| already owned by an open PR | #586, #893, #914, #938 |
28+
| upstream trackers | #92, #241, #417 |
29+
| awaiting reporter | #418, #796, #904 |
30+
31+
Zero closed on a partial fix. Every open issue now carries at least two labels.
32+
33+
## What the audit gates were worth
34+
35+
Twelve adversarial rounds across three units, nine FAIL.
36+
37+
On the plan (4 rounds, 3 FAIL): long context × Fast composition is impossible —
38+
OpenAI does not serve long context in Fast mode, so the `$7.80` figure
39+
described a request that cannot exist. The `-pro` aliases were unpriceable, so
40+
a tier row could never have been reached and the proposed test would have
41+
failed against the real estimator. `usesAdaptiveThinking()` was the wrong gate
42+
and would have required breaking a passing test to ship a 400.
43+
44+
On the #545 code (3 rounds, 2 FAIL): the gate silently missed
45+
`anthropic/claude-sonnet-5`. Then **my own remediation was worse than the bug** —
46+
normalizing with `lastIndexOf("/")` fixed the prefix case and broke the suffix
47+
case, and since the adaptive-wire predicate shares that parse, a slash-suffixed
48+
Sonnet 5 would have been sent obsolete manual `thinking.enabled` and 400d. A
49+
silent truncation traded for a hard failure, caught only because the reviewer
50+
re-ran the comparison across the full existing matrix rather than trusting the
51+
four ids I had listed.
52+
53+
Two of the plan blockers would have shipped code that passed CI and was wrong.
54+
55+
On the #915 code (5 rounds, 4 FAIL) the failures compounded, and each one was a
56+
smaller version of the same mistake:
57+
58+
1. Unknown plans failed **open** — an unfamiliar `plan_type` cleared a cooldown
59+
on evidence we could not interpret.
60+
2. The fix for that was an allowlist, which missed `prolite`.
61+
3. The allowlist was then the wrong *shape*: the snapshot carries 21 plan
62+
strings with 12 unclassified, and `CodexAccount.plan` is unrestricted. Every
63+
omission meant an account cooled **forever** — this unit's own defect,
64+
reintroduced as a typo-shaped hole. Replaced with the parser's binary rule,
65+
now shared by parsing, exhaustion, and recovery.
66+
4. The replacement test computed its expectation from the function under test,
67+
so ablating the rule to always-weekly still passed 18/18.
68+
69+
Three of those four were introduced by *fixing the previous one*. The lesson is
70+
not that lists need care; it is that a list was never the right answer where the
71+
domain is an open string.
72+
73+
Three tests also passed vacuously and were rebuilt: both lease-release cases
74+
asserted only that the cooldown survived, never that the lease returned (a
75+
stranded lease means that account is never probed again), and the fairness test
76+
spaced its passes inside the probe interval, so already-probed accounts dropped
77+
out on their own and the ordering was never exercised.
78+
79+
## Two hypotheses that did not survive
80+
81+
**#545's recorded cause was impossible.** The analysis said the prepended OAuth
82+
identity block consumed the classifier's 64 output tokens. The identity goes
83+
into the system prompt; `max_tokens` caps output. This repository had already
84+
rejected that theory once — `devlog/_fin/260728_bug_bundle_resolution/030_claude_system_dedup.md`
85+
abandoned an identity-dedup patch for the same reason — and it was re-derived
86+
anyway. The real cause was a round-trip loss where `disabled` and `omitted`
87+
collapsed to one representation, and omitted means thinking-**on**.
88+
89+
**#875's reopen was correct on the evidence and wrong on the facts.** The
90+
reporter tested `0b30283b6`; `git merge-base` puts it 115 commits before
91+
`5dd965a13`, the commit that fixes exactly that path. Left open with a re-test
92+
request rather than closed on an argument the reporter cannot check from their
93+
side.
94+
95+
## Carried contributor work
96+
97+
Six PRs cherry-picked with `-x`, authorship intact, no content changes:
98+
#939, #943, #944, #945 (@DevMello), #942 (@L14nY1Wang), #948 (@mushikingh).
99+
100+
Not carried, each for a stated reason: #947 conflicts with #942 on the same
101+
relay path and resolving that is its author's call; #933 is a draft with a
102+
failing gate; #928, #922, #940, #935 carry `CHANGES_REQUESTED`, and carrying a
103+
PR past requested changes routes around the review.
104+
105+
The source PRs stay open until #953 lands, so the direct path remains available
106+
if a maintainer prefers it.

0 commit comments

Comments
 (0)