Repository navigation
Commit aa30961
authored
Make the symbol-feed count and request-feed inventory size limits parameters, default unlimited (#675)
The runner refused more than 256 symbols or feeds and the two inventory
readers refused an inventory over 8 MiB, each as a fixed constant. They are
now options: --max-symbol-feeds on the runner, --max-inventory-bytes on the
runner and the inventory binder, forwarded by the entrypoint through
PINEFORGE_MAX_SYMBOL_FEEDS and PINEFORGE_MAX_INVENTORY_BYTES. Unset or 0 is
unlimited, a positive value is the limit, and anything else is a usage error.
With a limit set the typed refusals (too_many_symbols, too_many_feeds, binder
exit 10, the incomplete-inventory admission refusal) are unchanged and name the
actual limit.
The former values live in one opt-in set, selected by --tv-compat or
PINEFORGE_TV_COMPAT; an explicit limit, including 0, wins over it. Without it
nothing is capped. The key-length limit is a name check and is unchanged. The
two new runner options share the "--m" prefix, so the ambiguous-prefix
diagnostic now lists them too; its test pins the new text.
Documentation pages and anchors that cited the moved lines are updated. The
BARS-BIN-INPUT design note drops statements that no longer hold, named here so
the revert check can match them: "All source coordinates below refer to that
tree." ; "At most 256 symbols and 256 total extra feeds across the index." ;
"`_SYMBOL_FEEDS_MAX` docker/run_json.py:N;" ; "The old comment example lists
only CSV feeds; it is an example, not an extension restriction (comment's
`feeds` docker/run_json.py:N)." ; "The actual branch accepts binary feeds
(`suffix` docker/run_json.py:N)." The sentences were introduced by 91353de; the cite `_SYMBOL_FEEDS_MAX` is too
short to match by words, so it is named by that commit here as well. The first is replaced by a statement that
runner and entrypoint coordinates follow the working source; the second and
third by the parameter wording (default no limit); the last two are folded
into one sentence saying the loader accepts CSV and binary feed paths.
Unset or 0 means no limit; a positive value is a limit; anything else is a usage error. The previous fixed values (256 feeds, 8 MiB) are available only through the opt-in --tv-compat table, where an explicit value still wins.
## Ambiguous-prefix message (named behaviour change)
Registering `--max-symbol-feeds`, `--max-inventory-bytes` and `--tv-compat` adds them to the candidate lists of the runner parser's ambiguous-prefix refusal:
- `--t` now also names `--tv-compat` (`ambiguous option: --t could match --transpiled, --trade-start-ms, --tv-compat`);
- `--m` and `--ma` also name `--max-symbol-feeds` and `--max-inventory-bytes`.
The exit status (2), the failure code (`run_request_invalid`) and its args (`{"option":"arguments"}`) are unchanged. The `--m` text is pinned by `test_existing_ambiguous_prefix_keeps_its_error`; the `--t` and `--ma` texts are pinned by `test_the_limit_options_join_the_ambiguous_prefix_lists`. The changelog states the change.
## Disclosures
- One sentence of `docs/design/BARS-BIN-INPUT.md` ("source tree" note) was rewritten, and statements about the old fixed limits were replaced by the parameter wording.
- Two existing tests were adapted: the 256-feed refusal row now passes `max_symbol_feeds=256`, and the `--m` prefix row expects the two new option names after the frozen text.
- Still capped by default outside this change (not touched): `scripts/run_strategy.py` (256 requests-manifest entries) and the converter's 2 MiB summary cap (`docker/pineforge_sink_to_json.py`).
- Open follow-ups from the independent read (not in this PR): the failure-catalog `english` templates still say "more than 256"; the binder imports the limits table from the runner module; non-regular inventory files and very large positive limits are not guarded.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---
Commits:
* ed4468c
Make the symbol-feed count and request-feed inventory size limits parameters, default unlimited
The runner refused more than 256 symbols or feeds and the two inventory
readers refused an inventory over 8 MiB, each as a fixed constant. They are
now options: --max-symbol-feeds on the runner, --max-inventory-bytes on the
runner and the inventory binder, forwarded by the entrypoint through
PINEFORGE_MAX_SYMBOL_FEEDS and PINEFORGE_MAX_INVENTORY_BYTES. Unset or 0 is
unlimited, a positive value is the limit, and anything else is a usage error.
With a limit set the typed refusals (too_many_symbols, too_many_feeds, binder
exit 10, the incomplete-inventory admission refusal) are unchanged and name the
actual limit.
The former values live in one opt-in set, selected by --tv-compat or
PINEFORGE_TV_COMPAT; an explicit limit, including 0, wins over it. Without it
nothing is capped. The key-length limit is a name check and is unchanged. The
two new runner options share the "--m" prefix, so the ambiguous-prefix
diagnostic now lists them too; its test pins the new text.
Documentation pages and anchors that cited the moved lines are updated. The
BARS-BIN-INPUT design note drops statements that no longer hold, named here so
the revert check can match them: "All source coordinates below refer to that
tree." ; "At most 256 symbols and 256 total extra feeds across the index." ;
"`_SYMBOL_FEEDS_MAX` docker/run_json.py:N;" ; "The old comment example lists
only CSV feeds; it is an example, not an extension restriction (comment's
`feeds` docker/run_json.py:N)." ; "The actual branch accepts binary feeds
(`suffix` docker/run_json.py:N)." The sentences were introduced by 91353de; the cite `_SYMBOL_FEEDS_MAX` is too
short to match by words, so it is named by that commit here as well. The first is replaced by a statement that
runner and entrypoint coordinates follow the working source; the second and
third by the parameter wording (default no limit); the last two are folded
into one sentence saying the loader accepts CSV and binary feed paths.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
* 15dab60
Merge main into the runner limits branch
Takes main's run quota admission change; no conflict.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
* a334e0f
Name the ambiguous-prefix message change of the limit options and pin it
Registering --max-symbol-feeds, --max-inventory-bytes and --tv-compat adds
them to the candidate lists of the runner parser's ambiguous-prefix refusal:
--t now also names --tv-compat, and --m and --ma also name the two limit
options. The exit status, the failure code and its args are unchanged. The
changelog now states this, and a test pins the exact new text of the --t and
--ma refusals (the --m text is pinned by the existing prefix test).
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
* 59bd663
Merge main into the limits branch
Takes the output series limit parameter that landed on main. The runner
parser now parses through the output-series helper and then applies the
compatibility defaults of the limit options; the changelog keeps both
entries; the memory page keeps main's text with the run_json.py citations
re-pinned to the merged file.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
* 2ee8cf7
Native live startup test: the example exports engine layout stamp 2
The output series limit change raised PF_STREAM_ENGINE_LAYOUT_VERSION from 1
to 2 on purpose (the engine gained a member), and the module-layout assertion
in native_live_startup_e2e.py still pinned version 1, so the native-live legs
failed on main. The pin moves to 2: the normally linked native example must
still export the stamp, and it must still be exactly the version the runner
accepts; an accidental bump stays a test failure.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
---
Merge note (L6 super top): train T3 = CR-566 (#675) + the red-main fix for #677. #677 bumped PF_STREAM_ENGINE_LAYOUT_VERSION 1 -> 2 on purpose (include/pineforge/pineforge.h); the native-live startup test still pinned 1, so main 78f369e went red in post-merge native-live (TEST-BUG, triaged by ORG). The fix is a literal pin to 2 in tests/native_live_startup_e2e.py (2ee8cf7), deliberately not derived from the header so an accidental bump still fails. Merge gate on head 2ee8cf7: pineforge/verify 8/8 (rj-20261031t160031-f332e3, verdict cb43df2d), pineforge/parity success (sweep exp-engine-pr675-2ee8cf74-20261101, 7989 probes, cov lost 0, band1/band2 +0/-0, hard reg 0, leavers 0; verdict b61016c9), GKE PR CI green. Native-live on this exact head on spot boxes before merge: native shards 1-4 passed 1,550/1,550 with skipped-test names identical to main's GitHub run 41211922405; live-sanitizers and live-tsan 118/119 each, the one red being native_live_ratio_interval_e2e, a timing case already present on main (GitHub ran it at 149.30 s sanitizers / 67.33 s tsan, passing at its stdin-idle bound; our loaded boxes 154.5 s / 132.6 s failed; alone on the idle box it passed 3/3 at 65.7-66.0 s, 0 sanitizer reports). Carded CR-583: make that test load-proof (parameterized idle bound).1 parent 78f369e commit aa30961
22 files changed
Lines changed: 592 additions & 166 deletions
File tree
- docker
- docs
- design
- pages
- tests
- ownership
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
10 | 27 | | |
11 | 28 | | |
12 | 29 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
270 | 270 | | |
271 | 271 | | |
272 | 272 | | |
273 | | - | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
274 | 281 | | |
275 | 282 | | |
276 | 283 | | |
| |||
284 | 291 | | |
285 | 292 | | |
286 | 293 | | |
287 | | - | |
| 294 | + | |
| 295 | + | |
288 | 296 | | |
289 | 297 | | |
290 | 298 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
35 | | - | |
| 35 | + | |
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | 64 | | |
70 | 65 | | |
71 | 66 | | |
| |||
87 | 82 | | |
88 | 83 | | |
89 | 84 | | |
90 | | - | |
| 85 | + | |
91 | 86 | | |
92 | 87 | | |
93 | | - | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
94 | 92 | | |
95 | 93 | | |
96 | 94 | | |
97 | | - | |
| 95 | + | |
98 | 96 | | |
99 | | - | |
| 97 | + | |
100 | 98 | | |
101 | 99 | | |
102 | 100 | | |
| |||
175 | 173 | | |
176 | 174 | | |
177 | 175 | | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
178 | 180 | | |
179 | 181 | | |
180 | 182 | | |
| |||
184 | 186 | | |
185 | 187 | | |
186 | 188 | | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
187 | 193 | | |
| 194 | + | |
| 195 | + | |
188 | 196 | | |
189 | | - | |
| 197 | + | |
190 | 198 | | |
191 | 199 | | |
192 | 200 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
54 | 62 | | |
55 | 63 | | |
56 | 64 | | |
| |||
261 | 269 | | |
262 | 270 | | |
263 | 271 | | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
264 | 275 | | |
265 | 276 | | |
266 | 277 | | |
| |||
270 | 281 | | |
271 | 282 | | |
272 | 283 | | |
273 | | - | |
| 284 | + | |
| 285 | + | |
274 | 286 | | |
275 | 287 | | |
276 | 288 | | |
| |||
280 | 292 | | |
281 | 293 | | |
282 | 294 | | |
283 | | - | |
| 295 | + | |
284 | 296 | | |
285 | 297 | | |
286 | 298 | | |
| |||
293 | 305 | | |
294 | 306 | | |
295 | 307 | | |
| 308 | + | |
| 309 | + | |
296 | 310 | | |
297 | 311 | | |
298 | 312 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
73 | | - | |
74 | 73 | | |
75 | 74 | | |
76 | 75 | | |
| |||
157 | 156 | | |
158 | 157 | | |
159 | 158 | | |
160 | | - | |
| 159 | + | |
161 | 160 | | |
162 | 161 | | |
163 | 162 | | |
164 | 163 | | |
165 | | - | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
166 | 168 | | |
167 | 169 | | |
168 | | - | |
169 | | - | |
| 170 | + | |
| 171 | + | |
170 | 172 | | |
171 | 173 | | |
172 | 174 | | |
| |||
238 | 240 | | |
239 | 241 | | |
240 | 242 | | |
241 | | - | |
| 243 | + | |
| 244 | + | |
242 | 245 | | |
243 | 246 | | |
244 | | - | |
| 247 | + | |
| 248 | + | |
245 | 249 | | |
246 | 250 | | |
247 | 251 | | |
| |||
255 | 259 | | |
256 | 260 | | |
257 | 261 | | |
258 | | - | |
| 262 | + | |
259 | 263 | | |
260 | 264 | | |
261 | 265 | | |
| |||
281 | 285 | | |
282 | 286 | | |
283 | 287 | | |
284 | | - | |
| 288 | + | |
| 289 | + | |
285 | 290 | | |
286 | 291 | | |
287 | 292 | | |
| |||
312 | 317 | | |
313 | 318 | | |
314 | 319 | | |
315 | | - | |
| 320 | + | |
| 321 | + | |
316 | 322 | | |
317 | 323 | | |
318 | 324 | | |
| |||
0 commit comments