Skip to content

Commit 01516d1

Browse files
authored
Merge pull request #333 from Cor-Incorporated/fix/cor-local-limit-context-262144
fix(guardrails): cor-local limit.context を 262144 に(router -c 262144 と一致)
2 parents 3f3ed85 + 93a71d7 commit 01516d1

4 files changed

Lines changed: 14 additions & 11 deletions

File tree

‎packages/guardrails/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -130,9 +130,9 @@ OPENCODE_DISABLE_MODELS_FETCH=1 OPENCODE_DISABLE_AUTOUPDATE=1 opencode run --mod
130130

131131
`share` is already `disabled` in both guardrails profiles, so no extra flag is needed for that.
132132

133-
Cold-start note: the router loads models on first request. GLM in particular can block up to roughly 60 seconds before the first byte arrives — this is expected, not a hang. `options.headerTimeout` is set to 900000ms (15m) and `options.timeout` to `false` so that cold load doesn't trip the request-level timeout. `options.chunkTimeout` is set to 120000ms (2m) separately: once headers arrive, a stalled generation (no SSE chunk for 2 minutes) still aborts instead of hanging forever, since `timeout: false` alone does not bound the time between streamed chunks. All three models return `reasoning_content` (surfaced via each model's `interleaved: "reasoning_content"` declaration), structured `tool_calls`, and SSE streaming. Context window is 65536 tokens, max output 16384 tokens.
133+
Cold-start note: the router loads models on first request. GLM in particular can block up to roughly 60 seconds before the first byte arrives — this is expected, not a hang. `options.headerTimeout` is set to 900000ms (15m) and `options.timeout` to `false` so that cold load doesn't trip the request-level timeout. `options.chunkTimeout` is set to 120000ms (2m) separately: once headers arrive, a stalled generation (no SSE chunk for 2 minutes) still aborts instead of hanging forever, since `timeout: false` alone does not bound the time between streamed chunks. All three models return `reasoning_content` (surfaced via each model's `interleaved: "reasoning_content"` declaration), structured `tool_calls`, and SSE streaming. Context window is 262144 tokens, max output 16384 tokens.
134134

135-
`limit.context` (65536) must equal the router's `llama-server` launch flag `-c 65536` (ai-cluster repo). They were at 32768 through 2026-09-02; with the guardrails profile's AGENTS.md (15.6KB) plus system prompt and tool schemas already eating into the window, the mismatch made opencode auto-compact after every single step (`packages/opencode/src/session/compaction.ts`'s synthetic "Continue if you have next steps…"), and both `qwen3.8-27b` and `glm53-flash` looped re-planning without terminating even after fixing the target bug. If you change the router's `-c` value, update `limit.context` in both `managed/opencode.json` and `profile/opencode.json` to match — `model-whitelist.test.ts` pins both to 65536.
135+
`limit.context` (262144) must equal the router's `llama-server` launch flag `-c 262144` (ai-cluster repo, `config/llama-router/cor-local.ini.template`). They were at 32768 through 2026-09-02 and at 65536 from 2026-09-03 to 2026-09-07 (raised because a trivial two-line edit session on glm53-flash consumed 62% of the 64k window; opencode auto-compacts at `context - limit.output`, so the usable working window was only ~25K after the AGENTS.md/system prompt/tool schemas); with the guardrails profile's AGENTS.md (15.6KB) plus system prompt and tool schemas already eating into the window, the mismatch made opencode auto-compact after every single step (`packages/opencode/src/session/compaction.ts`'s synthetic "Continue if you have next steps…"), and both `qwen3.8-27b` and `glm53-flash` looped re-planning without terminating even after fixing the target bug. If you change the router's `-c` value, update `limit.context` in both `managed/opencode.json` and `profile/opencode.json` to match — `model-whitelist.test.ts` pins both to 262144.
136136

137137
Offline verification (no network): `bun run local:check-offline` runs the guardrails whitelist/paid/preview pin tests plus `scripts/sync-model-whitelist.py`'s consistency check across the three copies (managed, profile, `paid`).
138138

‎packages/guardrails/managed/opencode.json‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -454,7 +454,7 @@
454454
"reasoning": true,
455455
"interleaved": "reasoning_content",
456456
"limit": {
457-
"context": 65536,
457+
"context": 262144,
458458
"output": 16384
459459
}
460460
},
@@ -464,7 +464,7 @@
464464
"reasoning": true,
465465
"interleaved": "reasoning_content",
466466
"limit": {
467-
"context": 65536,
467+
"context": 262144,
468468
"output": 16384
469469
}
470470
},
@@ -474,7 +474,7 @@
474474
"reasoning": true,
475475
"interleaved": "reasoning_content",
476476
"limit": {
477-
"context": 65536,
477+
"context": 262144,
478478
"output": 16384
479479
}
480480
}

‎packages/guardrails/profile/opencode.json‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -480,7 +480,7 @@
480480
"reasoning": true,
481481
"interleaved": "reasoning_content",
482482
"limit": {
483-
"context": 65536,
483+
"context": 262144,
484484
"output": 16384
485485
}
486486
},
@@ -490,7 +490,7 @@
490490
"reasoning": true,
491491
"interleaved": "reasoning_content",
492492
"limit": {
493-
"context": 65536,
493+
"context": 262144,
494494
"output": 16384
495495
}
496496
},
@@ -500,7 +500,7 @@
500500
"reasoning": true,
501501
"interleaved": "reasoning_content",
502502
"limit": {
503-
"context": 65536,
503+
"context": 262144,
504504
"output": 16384
505505
}
506506
}

‎packages/guardrails/profile/plugins/model-whitelist.test.ts‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -135,14 +135,17 @@ describe("model whitelist stays consistent across its copies", () => {
135135
// (pytest green) but looped re-planning and never terminated. The router
136136
// (ai-cluster repo, llama-server launcher) moved to `-c 65536`; this pins
137137
// limit.context to the same value so the two sides cannot drift again.
138+
// 2026-09-07: the router moved again to `-c 262144` (a two-line config
139+
// edit on glm53-flash consumed 62% of the 64k window; all three GGUFs
140+
// have native context >= 262144 and the KV fits the coding budget).
138141
for (const id of ids) {
139-
test(`${id}: limit.context is 65536 (router -c) and limit.output is 16384 in both copies`, () => {
142+
test(`${id}: limit.context is 262144 (router -c) and limit.output is 16384 in both copies`, () => {
140143
expect(modelLimit(managed as Config, "cor-local", id)).toEqual({
141-
context: 65536,
144+
context: 262144,
142145
output: 16384,
143146
})
144147
expect(modelLimit(profile as Config, "cor-local", id)).toEqual({
145-
context: 65536,
148+
context: 262144,
146149
output: 16384,
147150
})
148151
})

0 commit comments

Comments
 (0)