Skip to content

Commit 5e04c78

Browse files
committed
docs: record PR 1196 final review
1 parent 908be9c commit 5e04c78

2 files changed

Lines changed: 222 additions & 0 deletions

File tree

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# PR 1196 Consensus Review
2+
3+
- Repository: `AltimateAI/altimate-code`
4+
- Pull request: [#1196](https://github.com/AltimateAI/altimate-code/pull/1196)
5+
- Review dates: 2026-08-30 through 2026-08-31
6+
- Final code candidate: `908be9cabb2b552c56cbd86537fbccb86ea5e0b2`
7+
- Mode: full Council review plus final independent remediation pass
8+
- Local verdict: **PASS**
9+
- Remote status: final commits still need to be pushed and fresh CI/bot review must finish
10+
11+
## Decision
12+
13+
PR #1196 is locally merge-ready under the selected product scope.
14+
15+
The feature prevents avoidable provider failures by reserving output only after the real request shape is known. The final code budgets system text, messages, finalized tool schemas, provider instructions, semantic media, dedicated input ceilings, context-expansion headers, and fixed reasoning options at both production request boundaries.
16+
17+
PDF accounting is intentionally approximate. The user selected a parser-free policy:
18+
19+
```text
20+
PDF allowance = max(32,768, decoded inline payload bytes)
21+
```
22+
23+
Remote URLs and provider file IDs receive the fixed allowance. Exact page expansion and tokenization remain provider-authoritative. The Council did not require exact PDF parsing because adding a parser would create a new document-processing/resource boundary without making provider token accounting exact.
24+
25+
## Original Council gate
26+
27+
The first three-round Council unanimously rejected the pre-repair candidate and required five concrete fixes:
28+
29+
1. Enforce `model.limit.input` as a hard estimated-input ceiling.
30+
2. Count every semantic media occurrence conservatively, including URL-backed and AI SDK v6 tool-result variants.
31+
3. Reconcile fixed reasoning budgets with every defined final output reservation.
32+
4. Distinguish repeated aliases from true cycles so repeated schemas/media are counted per transport occurrence.
33+
5. Add behavioral proof that both AI SDK and native request paths send finalized tools, clamped output, and reconciled reasoning together.
34+
35+
All five are present in the final branch with focused regressions.
36+
37+
## Additional review findings
38+
39+
Post-Council review found and repaired:
40+
41+
- request-header precedence drift around the Anthropic 1M context flag;
42+
- unsupported-media estimation before canonical projection;
43+
- media-looking ordinary text being discounted as binary content;
44+
- small-model output floors above the model's own reservation;
45+
- safety-margin loss near the context boundary;
46+
- non-JSON provider-option loss during reasoning reconciliation;
47+
- raw PDF page-marker trust.
48+
49+
Each repair was centralized in the shared output-budget module or the existing pure media projection, then exercised through both request boundaries.
50+
51+
## Why the PDF parser experiment was rejected
52+
53+
An intermediate branch attempted structural page counting with `pdf-lib`. Independent reviewers found two blockers:
54+
55+
- the fixed 100-page/500 KB policy still undercounted valid many-page requests on supported 1M-context models;
56+
- compressed object streams could expand substantially in-process before request admission, while the input gate bounded only compressed bytes.
57+
58+
The parser experiment was reverted. The dependency and transitive lock entries are absent from the final tree, estimator APIs are synchronous again, and tests describe the fixed allowance as parser-free.
59+
60+
Batching the same parser would not remove its decompression/traversal boundary. A subagent is useful for development review, but it is not a runtime memory, CPU, or trust boundary. The smallest safe scope is therefore the crude local estimate plus provider enforcement.
61+
62+
## Final independent remediation review
63+
64+
Two independent live Council seats re-reviewed exact head `908be9cabb` after the parser removal.
65+
66+
### Feynman seat — PASS
67+
68+
- Verified exact final head.
69+
- Confirmed `pdf-lib`, async parsing, page regex/policy, and transitive lock entries are absent.
70+
- Confirmed lazy estimation is not evaluated when `maxOutputTokens` is omitted.
71+
- Confirmed both request boundaries clamp after headers, tools, instructions, and media projection are finalized.
72+
- Re-ran provider, native, stream, typecheck, diff, and strict marker checks successfully.
73+
74+
### Musashi seat — PASS
75+
76+
- Confirmed the parser experiment is cleanly reverted.
77+
- Confirmed the final code differs from the last pre-parser safe candidate only in the explicit parser-free comment and test naming.
78+
- Confirmed synchronous lazy estimation and parity across both callers.
79+
- Confirmed dependency cleanup, focused tests, typecheck, and diff checks pass.
80+
81+
### Degraded seat
82+
83+
The original chairman seat was rejected twice by the service safety filter because its retained conversation context included the earlier parser stress case. It produced no contrary code finding on the final head. The thread limit prevented replacing that retained seat with a new fourth thread.
84+
85+
Consensus therefore rests on two independent live PASS votes, the primary review, the complete changed-file inspection, and a sealed zero-finding Codex Security remediation scan. The degraded seat is disclosed rather than silently counted as agreement.
86+
87+
## Final verification
88+
89+
- Focused provider, AI-SDK stream, native request, and upstream bridge suites: **413 passed, 11 skipped, 1 existing todo, 0 failed**.
90+
- Repository typecheck: **13/13 successful**.
91+
- Strict changed-file marker validation: passed.
92+
- Required-marker inventory: **35/35**.
93+
- Frozen lockfile install: **1,295 installs across 1,390 packages, no changes**.
94+
- Targeted oxlint: **212 warnings, 0 errors**.
95+
- Prettier: all changed files pass except `provider/provider.ts` and `provider/transform.ts`; both fail identically on `origin/main`.
96+
- `git diff --check origin/main...HEAD`: passed.
97+
- Final Codex Security remediation scan: complete coverage, **0 findings**.
98+
99+
## Residual limitations
100+
101+
- Compact or dense PDFs can be underestimated locally and rejected by the provider.
102+
- Exact tokenizer parity across providers is not claimed.
103+
- No live request was made against every provider/context combination.
104+
- Broader tokenizer calibration and document-ingestion architecture belong in follow-up work, not this PR.
105+
106+
## Merge gate
107+
108+
The local recommendation is **merge after remote completion**, provided:
109+
110+
1. the final commits are pushed without overwriting concurrent remote work;
111+
2. every review thread is answered or resolved against the new head;
112+
3. fresh required CI and bot reviews are green; and
113+
4. no new critical finding appears on the pushed head.
114+
115+
Do not merge merely on the strength of the stale green checks attached to `9039a178c0`.
116+
117+
## Execution reliability
118+
119+
- Intended panel seats: 3
120+
- Final live PASS seats: 2
121+
- Degraded seats: 1 (service filter)
122+
- Offline seats: 0
123+
- Final retries of degraded seat: 1
124+
- Fallback: primary reviewer completed the full diff and security reconciliation
125+
- Code-discovery source: repository knowledge graph plus exact immutable Git diffs
126+
127+
---
128+
129+
- `schema_version: 1`
130+
- `mode: full`
131+
- `panel_size: 3`
132+
- `final_live_votes: 2`
133+
- `final_pass_votes: 2`
134+
- `degraded_seats: 1`
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# PR 1196 Security Review
2+
3+
- Repository: `AltimateAI/altimate-code`
4+
- Pull request: [#1196](https://github.com/AltimateAI/altimate-code/pull/1196)
5+
- Review dates: 2026-08-30 through 2026-08-31
6+
- Final code candidate: `908be9cabb2b552c56cbd86537fbccb86ea5e0b2`
7+
- Scan mode: chained immutable branch-diff reviews
8+
- Final coverage: complete
9+
- Findings remaining on the final candidate: **0**
10+
11+
## Outcome
12+
13+
The final candidate removes local PDF parsing entirely. PDF media now uses a deliberately crude, parser-free estimate:
14+
15+
```text
16+
max(32,768 tokens, decoded inline payload bytes)
17+
```
18+
19+
Remote URLs and provider file IDs receive the fixed 32,768-token allowance. The estimate is monotonic in locally observable payload size, ignores untrusted page metadata, and does not decompress or traverse PDF structure. The configured provider remains authoritative for exact tokenization and for unusually compact or dense documents.
20+
21+
This is the selected product boundary, not an attempt at exact PDF accounting. A compact or dense PDF can still be underestimated locally and rejected by the provider. That residual is an acknowledged reliability limitation; it is not a local parser, authorization, confidentiality, integrity, or shared-service vulnerability.
22+
23+
## Scan chain
24+
25+
The review used immutable ranges so every material repair was independently reconciled.
26+
27+
| Stage | Immutable range/candidate | Result |
28+
| ---------------------------- | -------------------------------------------------- | ------------------------------------------------------------ |
29+
| Initial PR review | current `main` through remote PR head `9039a178c0` | Complete coverage; no remaining finding in that candidate |
30+
| Media hardening supplement | `9039a178c0..e37b7a974d` | One validated Low finding: raw PDF page-marker amplification |
31+
| Marker fix | `ac7346e767` | Removed lexical page-marker trust |
32+
| Structural parser experiment | `ac7346e767..858c7b1dab` | Two validated Low findings; experiment rejected |
33+
| Parser-free remediation | `858c7b1dab..908be9cabb` | Complete coverage; **0 findings** |
34+
35+
The final remediation scan was sealed once as scan `80591880-0a17-454c-b312-92c32a38f5ff`. Its authoritative result contains six reviewed surfaces, no deferred work, no open question, and zero findings.
36+
37+
## Findings discovered and resolved
38+
39+
### 1. Raw page markers could poison local admission
40+
41+
An intermediate estimator scanned PDF bytes for lexical `/Type /Pages /Count` text. A marker in a comment, literal string, stream, or unreachable object could therefore inflate the local estimate and reject a valid user turn before transport.
42+
43+
The fix removed page-marker scanning. Regression tests cover comment, string, stream, and unreachable-object variants across string, base64/data URL, `Uint8Array`, and `ArrayBuffer` payload shapes.
44+
45+
### 2. In-process parsing could amplify compressed object streams
46+
47+
The structural-parser experiment passed attacker-influenced PDF bytes to `pdf-lib` inside the synchronous request path. A bounded reproduction used a 51,430-byte PDF whose unreachable compressed object stream expanded to 50 MiB while loading and increased process RSS by roughly 88 MiB. The byte gate limited compressed input, not decompressed output, traversal work, or memory.
48+
49+
The final candidate removes `pdf-lib`, `PDFDocument.load`, all structural page traversal, and the parser's transitive lockfile entries. Graph-augmented source search found no remaining runtime parser reference.
50+
51+
### 3. The parser policy still disagreed with supported long-context requests
52+
53+
The experiment coupled a 100-page fallback to a 500 KB parser ceiling. A valid 600-page PDF on a 1M-context request could sit just above that byte ceiling, receive only a byte-sized estimate, and preserve a large output reservation. Structural parsing therefore did not make the local estimate authoritative; it merely added a new resource boundary.
54+
55+
The final policy removes the 100-page claim instead of replacing it with a larger parser. Exact page expansion is explicitly delegated to the provider.
56+
57+
## Final trust and data flow
58+
59+
Both production request paths use the same sequence:
60+
61+
1. Finalize messages, tools, provider instructions, headers, and plugin-selected output reservation.
62+
2. If no output reservation or credible limit exists, return without serializing the prompt.
63+
3. Lazily estimate text, schemas, semantic media allowances, and decoded inline payload size.
64+
4. Enforce a dedicated input limit when declared.
65+
5. Clamp the output reservation against the shared context window with a safety margin.
66+
6. Reconcile fixed reasoning budgets with the final reservation.
67+
7. Send through the AI SDK or native transport.
68+
69+
Codebase graph tracing found exactly two production callers of the centralized clamp: `session/llm.ts` and `session/llm/request.ts`.
70+
71+
## Verification
72+
73+
- Focused provider, AI-SDK stream, native request, and upstream bridge suites: **413 passed, 11 skipped, 1 existing todo, 0 failed**.
74+
- Repository typecheck: **13/13 tasks successful**.
75+
- Strict changed-file marker validation: passed.
76+
- Required-marker inventory: **35/35**.
77+
- Frozen lockfile install: **1,295 installs across 1,390 packages, no changes**.
78+
- Targeted oxlint: **0 errors**; warnings remain repository debt.
79+
- Prettier: all changed files pass except `provider/provider.ts` and `provider/transform.ts`; both fail identically at `origin/main`, so no unrelated whole-file rewrite was introduced.
80+
- `git diff --check origin/main...HEAD`: passed.
81+
- Final Codex Security remediation scan: complete, **0 findings**.
82+
83+
## Operational caveats
84+
85+
- TAC advisory was attempted once earlier in the PR workflow and was unavailable; it was not retried.
86+
- No live provider request or full interactive UI replay was performed.
87+
- Provider-side rejection remains possible for compact or unusually dense PDFs because the local estimate is intentionally crude.
88+
- The remote PR still needs the final commits pushed, current review threads reconciled, and fresh CI/bot results before a merge recommendation.

0 commit comments

Comments
 (0)