Skip to content

Commit ddd82d4

Browse files
terisukeclaude
andcommitted
fix(guardrails): remove OpenRouter eval-only restriction (#24)
Empty the evals Set so all configured providers are available for standard work. Guard both eval-related gate checks with evals.size > 0 to prevent provider-eval from being blocked when no eval providers are defined. Update tests accordingly. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
1 parent 67e659f commit ddd82d4

3 files changed

Lines changed: 9 additions & 6 deletions

File tree

‎packages/guardrails/profile/AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,4 +10,4 @@
1010
- Treat `.opencode/guardrails/` as plugin-owned runtime state, not a manual editing surface.
1111
- Use `implement` as the guarded default primary agent. Route review, ship, and handoff work through the packaged `/review`, `/ship`, and `/handoff` commands instead of freeform release flows.
1212
- Keep review paths read-only. If a workflow needs edits, return to `implement` or a project-local implementation agent instead of widening the review agent.
13-
- Keep provider admission explicit. Standard confidential-code work stays on the admitted `zai` and `openai` lane; OpenRouter-backed evaluation belongs on `provider-eval` or `/provider-eval` only.
13+
- All configured providers are available for standard work. The `provider-eval` agent and `/provider-eval` command remain available for dedicated evaluation workflows.

‎packages/guardrails/profile/plugins/guardrail.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -202,7 +202,7 @@ export default async function guardrail(input: {
202202
worktree: string
203203
}, opts?: Record<string, unknown>) {
204204
const mode = typeof opts?.mode === "string" ? opts.mode : "enforced"
205-
const evals = new Set(["openrouter"])
205+
const evals = new Set<string>([])
206206
const evalAgent = "provider-eval"
207207
const conf = true
208208
const denyFree = true
@@ -349,10 +349,10 @@ export default async function guardrail(input: {
349349
const agent = str(data.agent)
350350
if (!provider) return
351351

352-
if (evals.has(provider) && agent !== evalAgent) {
352+
if (evals.size > 0 && evals.has(provider) && agent !== evalAgent) {
353353
return `${provider} is evaluation-only under confidential policy; use ${evalAgent}`
354354
}
355-
if (agent === evalAgent && !evals.has(provider)) {
355+
if (evals.size > 0 && agent === evalAgent && !evals.has(provider)) {
356356
return `${evalAgent} is reserved for evaluation-lane providers`
357357
}
358358

‎packages/opencode/test/scenario/guardrails.test.ts‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -373,6 +373,7 @@ test("guardrail profile enforces provider admission lanes", async () => {
373373

374374
const evalModel = openrouter.models["openai/gpt-5.4-mini"]
375375

376+
// With evals set empty, openrouter is NOT evaluation-only, so implement can use it
376377
await expect(
377378
Plugin.trigger(
378379
"chat.params",
@@ -383,8 +384,9 @@ test("guardrail profile enforces provider admission lanes", async () => {
383384
},
384385
{ temperature: undefined, topP: undefined, topK: undefined, options: {} },
385386
),
386-
).rejects.toThrow("evaluation-only")
387+
).resolves.toEqual({ temperature: undefined, topP: undefined, topK: undefined, options: {} })
387388

389+
// With evals set empty, provider-eval is open to any provider
388390
await expect(
389391
Plugin.trigger(
390392
"chat.params",
@@ -395,8 +397,9 @@ test("guardrail profile enforces provider admission lanes", async () => {
395397
},
396398
{ temperature: undefined, topP: undefined, topK: undefined, options: {} },
397399
),
398-
).rejects.toThrow("reserved for evaluation-lane providers")
400+
).resolves.toEqual({ temperature: undefined, topP: undefined, topK: undefined, options: {} })
399401

402+
// Whitelist-based admission still blocks unadmitted models on openrouter
400403
await expect(
401404
Plugin.trigger(
402405
"chat.params",

0 commit comments

Comments
 (0)