Skip to content

Commit 7186191

Browse files
DaubnerFedelauna
andauthored
Blanket auto-deny for unapproved commands (hands-free mode) (Zoo-Code-Org#1760)
* initial fix for issues Zoo-Code-Org#1240 and Zoo-Code-Org#505 * 1st round of fixes * fixed comments * increase test coverage * revert: remove Windows shell invocation from stryker-diff * fix: address CodeRabbit review on tool-policy prompt unification * fix(test): correct apiModelId in generateSystemPrompt state mock * drop use_mcp_tool from policy when no MCP tool is permitted * code hardening * initial implementation of blanket auto-deny commands * bound model fetch with timeout, typed provider state test doubles * cover preview model fetch timeout path with tests * pin completion-time history save ordering with unit tests * poll history length in restart e2e to tolerate atomic write window * share one model-info snapshot per request between prompt and tools * resolve provider state once before the MCP wait getSystemPrompt read provider state twice: once for the MCP gate and again after the hub wait. When the caller threaded no state, the two reads could observe different snapshots. Hoist the fallback resolution to the top of the call so the prompt and the tool guidance share one snapshot on the unthreaded path. Type the test harness getSystemPrompt signature with ProviderState and ModelInfo instead of unknown, and align the affected test title and comments with the single-read behavior. * cover the undefined provider state path in the system prompt tests The provider state read can resolve to nothing even while the provider reference stays alive. Add a test for that case so the system prompt call keeps receiving undefined disabledTools instead of failing. * reuse one model-info snapshot per request and honor cancellation Request construction re-read model metadata twice after the streaming turn's bounded fetch; thread the captured snapshot through attemptApiRequest so prompt assembly, context sizing, and tool arrays agree on a single view, resolving the fallback only when no snapshot was supplied. A cancellation that lands during a request's waits now stops the request before any tool array, abort controller, or provider call is issued for it. * pin the retry count the request seam receives The empty-response retry test now asserts that the retry iteration reaches attemptApiRequest with its own incremented attempt count (second call, retryAttempt 1), instead of only checking the resulting conversation history. * refactor(task): require callers to thread provider state into system prompt build getSystemPrompt no longer falls back to re-reading provider state; the provider-state snapshot parameter is now required. An explicit undefined declares that the caller's own read came back empty because the provider was already gone, and the prompt then resolves from defaults. The prompt and the request's runtime tool array now resolve from a single snapshot by construction rather than by caller convention. Behavior is unchanged on all reachable paths. Task.spec.ts grows from 128 to 129 tests to cover the required-parameter contract. * fix(api): cancel abandoned model-metadata waits via AbortSignal The bounded metadata waits in Task.safeEnsureModelFetched and the system prompt preview cleared their timer but left the handler-side promise waiting on the model-catalog fetch. The ApiHandler contract now threads an optional AbortSignal through ensureModelFetched(): RouterProvider settles the waiter with a rejection when the signal aborts, so an abandoned or cancelled caller detaches instead of parking a promise on the shared fetch (which keeps running for other waiters and still populates the cache, by design). The task aborts its waiter both when the 5s bound expires and when cancelCurrentRequest runs (cancel and dispose paths); the preview aborts at its bound and on completion. The task-lifecycle doc's table padding was also reconciled with the PR base: the remaining diff there is now only prettier's column re-padding, which the repo's own pre-commit formatter enforces. * test(api): cover abort-signal detach paths and thread request model snapshot Mutation-diff gate kills (PR Zoo-Code-Org#1505): - zoo-gateway: signal-aware ensureModelFetched tests for the fetch-wins and fetch-rejects branches (block/CallExpression NoCoverage), an addEventListener spy pinning the { once: true } options, and paired add/remove listener assertions pinning the abort event name on both detach sites (StringLiteral mutants). - Task: ownership-guard tests for metadataFetchAbortController (clear on own completion, leave a replaced controller in place). - generateSystemPrompt: signal-capture tests pinning the timeout-bound and finally-block controller.abort() detaches (CallExpression mutants). CodeRabbit: thread the request model-info snapshot into buildCleanConversationHistory so preserveReasoning resolves from the same per-request snapshot as the prompt and tool arrays, plus regression tests. No Stryker-disable directives were needed; all 14 mutants are killed behaviorally. * test(webview): pin alwaysDenyUnapprovedCommands persistence through updateSettings * chore: tighten blanket auto-deny comments and test names for accuracy * test: remove out-of-scope Task history-persistence spec Delete src/core/task/__tests__/Task.history-persistence.spec.ts (+407, branch-only) per the ratified removal set (ruling 2026-09-20b; cycle1-scope section 3 row 1; plan section 2.4 item #1). Pre-delete evidence (plans/1569/tmp/b2-removal-evidence.md): grep 'history-persistence' returns only the file's own header self-reference -- no importers; vitest collects the spec by glob. N_blocks = 12 grep-lines (7 it + 5 describe). T6 broad-suite delta vs merged head c1378c6: 94->93 files, 1533->1526 passed, 4 skipped -- exactly the deleted spec's contribution, zero collateral. warnProtocolToolOverrides grep-zero re-confirmed; eslint-suppressions byte-identical (no increase); cast baseline 3673->3670. * test(auto-deny): pin AutoApprovalContext forwarding seam and fix stale comments F1: rewrite the queue-shortcut comment in ask-auto-deny.spec.ts to a timeless statement (drop the unresolvable past-tense claim about a second bypass path). F2: fix the self-referential docblock on CommandDecisionDetail in auto-approval/commands.ts; the relation is now stated relative to getCommandDecision, matching the inverse docblock on getCommandDecisionDetailed. F3: pin the widened askApproval -> cline.ask -> checkAutoApproval forwarding chain that no test previously referenced: - presentAssistantMessage-auto-deny.spec.ts +2 positional-forwarding cases (tool_use and MCP askApproval closure copies) - ask-auto-deny.spec.ts +1 case pinning Task.ask's dcgDecision forwarding into the policy decision Each of the three forwarding sites is mutation-killed by these cases. Gates at fix head: G1 261/14, auto-deny spec 6/1, G2 82/1, G3 6, G4 26, G6 1529 passed / 4 skipped / 93 files, tsc 0, eslint touched files 0 warnings, suppressions byte-identical, test blocks +3 added / 0 deleted, casts 3670 -> 3672 (+2, both the file's established per-call mockTask-as-Task idiom in the two mandated new tests). * test(webview): visual snapshot for blanket auto-deny checkbox The blanket auto-deny setting makes command execution fail closed when command auto-approval is on: explicitly approved commands run, anything else is denied without prompting, and the denial reason goes back to the model. The settings row that controls it had no visual regression coverage. Add a Story Gallery snapshot test for the Auto-approve settings panel with the blanket auto-deny checkbox visible and checked, plus its story registration and baselines for the four supported themes (dark, light, high-contrast, high-contrast-light). The baselines were generated in the Playwright container image pinned by docker-compose.visual.yml and are committed so the surface is protected against future drift. The setting behavior itself landed in earlier commits of this branch; this commit only adds the visual coverage. * test(core): unit coverage for blanket auto-deny reasons and edge cases Add unit coverage closing this PR's patch coverage gap: per-kind denial reason strings with fallbacks and rule id suffix, rule_id payload pins, approval edge cases. Three defensive fallback branches are unreachable by proof and stay uncovered. No production changes. * test(core): align auto-deny test fixtures with policy emission shapes The auto-deny fixtures attached a DCG rule id to not_allowlisted and denylist denials, a shape checkAutoApproval never emits, so the rule forwarding assertions could pass without covering the DCG path. The fixtures now mirror the emitted shapes: non-DCG denials carry no rule id and assert none reaches the tool payload, while new dcg-denial tests pin rule forwarding in both askApproval copies. The mocked tool handlers also capture askApproval's return value and pin it to false on the deny path instead of discarding it. This addresses the automated review comments on the branch's test diff. * fix(core): deny verdictless DCG command asks with a retryable reason A verdictless command ask under an enabled Destructive Command Guard used to auto-approve. That state is a guard inconsistency, not a guard decision: it now denies with a guard_unavailable reason stating that the command did not run, that it was not denied by policy, and that the model may retry it. Shell-expansion denials now name the offending command and drop the approval cue that hands-free mode cannot honor. Guard-disabled and verdict-carrying flows are unchanged. * test(webview): cover blanket auto-deny toggle save round-trip Saving the settings view posts the blanket auto-deny toggle, and an unset toggle is coerced to false. The local settings buffer holds user edits until Save, so the tests assert both the buffered state and the persisted payload. Addresses automated-review feedback on missing round-trip coverage for the setting. * fix(task): re-check auto-approval policy when queued answers resume an ask An ask answered from the message queue applied the auto-approval decision frozen when the ask was created: the queued path skipped checkAutoApproval, so a settings change made while the ask was pending (blanket deny switched on, or auto-approval switched off) had no effect and the queued answer could act as approval for a command the new policy would deny. Both queued-answer consume sites in Task now re-read current settings and re-run checkAutoApproval before honoring the answer. When the re-check denies, the path matches the ask-time denial: same bail-out and reason, and the queued message's claim is released so the answer is not lost to the denied ask. The queue guard on that path changed too. It tested only whether the queue was non-empty, so a message still claimed by a previous ask, or left unclaimed after a denied-command ask, counted as an approval source for the next ask. The guard now uses a new hasUnclaimed() on MessageQueueService, and Task latches blanketDeniedCommandThisTurn so no later ask in the same turn can consume a leftover message as approval. A guard_unavailable auto-deny detail now surfaces as a retryable tool_error instead of the terminal toolAutoDenied result: the kind marks a guard-state inconsistency, not a policy decision, and the model should be able to retry rather than see the turn denied. Tests: flip-on regressions at both consume sites, bail-out and claim-release tests, hasUnclaimed coverage, and a spec pinning the auto-deny reason text in the tool-response prompts. * fix(tools): re-check command policy before an approved command runs An approved execute_command request ran on the policy snapshot taken before the ask: engaging blanket auto-deny while the approval prompt was open had no effect, and the command still reached the terminal. After approval the tool now re-reads settings and, with blanket auto-deny engaged, re-runs checkAutoApproval before the shell starts. A fresh deny is honored: a blanket cause returns the same auto-denied response as the ask-time denial and marks the turn, so a queued message left by the denial cannot answer a later ask; a guard-state inconsistency (guard_unavailable) or a deny without a reason returns a retryable tool_error instead, since neither is a policy decision. The blanket-deny derivation no longer lives in two places. ExecuteCommandTool calls isBlanketDenyEngaged and consults BLANKET_DENY_AUTO_DENY_KINDS, both now exported from Task and already used by the ask-time snapshot and the queued-answer re-checks, so the decision points cannot drift apart. The protected-prompt flag sent with the command ask is derived from the same snapshot instead of hardcoded, and is forwarded to the execute-time re-check so its decision is computed against the ask the user actually answered. Tests: blanket deny engaging during the approval dwell is denied at execute time; a guard flip during the dwell returns a retryable error without marking the turn; a user Deny landing while the queued-answer re-check awaits is not overwritten; a denial at the queue drain marks the turn and blocks the follow-up ask. * test(tools): cover blanket-deny kinds missing from the auto-deny spec The spec pinned only dcg and not_allowlisted. dangerous_substitution and malformed_command now have regression tests at the presentAssistantMessage layer: each denies without executing the command and without aborting the turn. * fix(task): arm ask status timers from all release paths An approved ask that gets handed back to the user - because a fresh command-policy read comes back denying, or because its claim release lands while other messages are queued - left the interactive, resumable, and idle status timers unarmed. The only arm site was gated on `isStatusMutable`, which is computed once before the queue claim and stays false while the ask waits on the user, so hands-free and API consumers saw no interaction signal for a prompt that was pending. The arming now lives in an idempotent helper that every release path calls; the timers re-check liveness at fire time and are torn down in the ask's finally block, so a late timer cannot emit for an answered, superseded, or aborted task. The post-approval command-policy read awaited `provider.getState()` with no abort channel, so an abort could leave `ask()` blocked on the pending read. The read now races a 100 ms abort watcher and settles within one tick of an abort. The auto-deny settings checkbox handler typed its event as `any`; it now narrows `Event | FormEvent<HTMLElement>` to an `HTMLInputElement`. Five new regression tests cover each release path, the single-emit idempotence, and the abort race; each was verified to fail against the pre-fix code. * fix(settings): read blanket auto-deny toggle from the checkbox target The blanket auto-deny checkbox handler only accepted a change event whose target is an HTMLInputElement. The toolkit checkbox is a shadow-DOM custom element, so its change events reach the wrapper retargeted to the host <vscode-checkbox> element, which exposes `checked` without being an input; the check swallowed those events and the toggle never updated the setting. Read the target structurally instead: a local type predicate accepts any target carrying a boolean `checked` and declines the rest. Both the host element and a directly rendered input now update the setting, and a target without `checked` is declined rather than writing `undefined` through. Two regression tests pin the behavior: one fires a change event retargeted to a non-input host element and asserts the setting write, one fires an event whose target carries no `checked` and asserts no write. The existing settings specs cover the toggle round-trip through Save unchanged. * fix(task): enforce blanket deny at queue claims and policy re-checks Follow-up fixes to the blanket command denial work, covering three behaviors around the message queue and the execute-time policy re-check. Claiming a queued message now respects the per-turn denial latch. A blanket denial leaves its queued message in place to be answered later; previously the next ask could still claim that message at the immediate site, and the claim forced a manual prompt even when the auto-approval policy would have answered the ask itself - a hands-free session could stall on a prompt no user needed to see. The claim is now gated by the same latch the drain path uses, so the policy decision stands and the message stays queued. The immediate execute-time policy re-check is now abort-safe. That re-check ends in an uncancellable provider read, so an abort landing while it was pending could leave ask() blocked and the queued message claimed. The wait now races an abort watcher, and one finally releases the claim on the abort, throw, and release outcomes alike - the shape the drain path already used. A command approved while its prompt was open can no longer execute past a blanket deny. The execute-time re-check now re-derives command protection from the fresh policy state instead of reusing the flag captured when the prompt was shown: if blanket denial was engaged in the meantime, the fresh derivation is unprotected, so the denial is evaluated. Any re-check result other than an explicit approval is also handled fail-closed - the command is not executed, and the tool returns a retryable error instead. Adds five regression tests: two for the queue-claim gate and the abort-safe re-check (ask-auto-deny.spec.ts), three for the execute-time re-check (executeCommandTool.spec.ts). * fix(auto-approval): cancel queued-command policy re-check on task abort * fix(ci): treat negative v8 branch deltas as uncovered when merging lcov * fix(ci): keep merge-lcov.mjs out of the mutation scope and pin its clamp boundaries fe30882 pushed the PR diff's extension-package executable line count to 503, tripping the mutation gate's MAX_CHANGED_LINES=500 hard failure (Stryker never ran). CI mutation testing is advisory for Survived/NoCoverage mutants, so the regression is the scope cap itself, not surviving mutants. - stryker-diff: exclude src/scripts/merge-lcov.mjs via the existing excludedPaths convention (CI build/coverage tooling, same class as src/esbuild.mjs; covered by src/scripts/__tests__/merge-lcov.spec.mjs), bringing the extension scope back to 497 lines; pin the exclusion in the packageForPath manifest test. - merge-lcov.spec: pin parseBranchCount boundaries (-1 clamps, 0 stays, positive passes through, union prefers the positive lane) and keep non-integer BRDA rejection pinned. * test(auto-approval): pin blanket re-check approve path actually executes the command --------- Co-authored-by: edelauna <[email protected]>
1 parent fcc51b5 commit 7186191

61 files changed

Lines changed: 4374 additions & 112 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎packages/types/src/__tests__/global-settings.test.ts‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import {
2+
DEFAULT_ALWAYS_DENY_UNAPPROVED_COMMANDS,
23
DEFAULT_DESTRUCTIVE_COMMAND_GUARD_ENABLED,
34
GLOBAL_SETTINGS_KEYS,
45
globalSettingsSchema,
@@ -20,3 +21,20 @@ describe("destructive command guard global setting", () => {
2021
expect(() => globalSettingsSchema.parse({ destructiveCommandGuardEnabled: "true" })).toThrow()
2122
})
2223
})
24+
25+
describe("alwaysDenyUnapprovedCommands global setting", () => {
26+
it("is opt-in by default", () => {
27+
expect(DEFAULT_ALWAYS_DENY_UNAPPROVED_COMMANDS).toBe(false)
28+
})
29+
30+
it("accepts and exposes the persisted setting", () => {
31+
expect(globalSettingsSchema.parse({ alwaysDenyUnapprovedCommands: true })).toEqual({
32+
alwaysDenyUnapprovedCommands: true,
33+
})
34+
expect(GLOBAL_SETTINGS_KEYS).toContain("alwaysDenyUnapprovedCommands")
35+
})
36+
37+
it("rejects non-boolean setting values", () => {
38+
expect(() => globalSettingsSchema.parse({ alwaysDenyUnapprovedCommands: "true" })).toThrow()
39+
})
40+
})

‎packages/types/src/global-settings.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,15 @@ export const DEFAULT_DIFF_FUZZY_THRESHOLD = 1.0
4949

5050
export const DEFAULT_DESTRUCTIVE_COMMAND_GUARD_ENABLED = false
5151

52+
/**
53+
* Whether commands that are not explicitly auto-approved are automatically
54+
* denied (with a structured reason sent to the model) instead of prompting the
55+
* user. Opt-in: by default, unapproved commands still ask for confirmation.
56+
* Only engages when command auto-approval (`autoApprovalEnabled` +
57+
* `alwaysAllowExecute`) is on.
58+
*/
59+
export const DEFAULT_ALWAYS_DENY_UNAPPROVED_COMMANDS = false
60+
5261
/**
5362
* Terminal output preview size options for persisted command output.
5463
*
@@ -154,6 +163,14 @@ export const globalSettingsSchema = z.object({
154163
alwaysAllowSubtasks: z.boolean().optional(),
155164
alwaysAllowExecute: z.boolean().optional(),
156165
destructiveCommandGuardEnabled: z.boolean().optional(),
166+
/**
167+
* Blanket auto-deny for unapproved commands. When true (and command
168+
* auto-approval is engaged), every command that is not explicitly
169+
* auto-approved is automatically denied with a structured reason delivered
170+
* to the model, instead of prompting the user.
171+
* @default false
172+
*/
173+
alwaysDenyUnapprovedCommands: z.boolean().optional(),
157174
alwaysAllowFollowupQuestions: z.boolean().optional(),
158175
followupAutoApproveTimeoutMs: z.number().optional(),
159176
allowedCommands: z.array(z.string()).optional(),

‎packages/types/src/vscode-extension-host.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -283,6 +283,7 @@ export type ExtensionState = Pick<
283283
| "alwaysAllowFollowupQuestions"
284284
| "alwaysAllowExecute"
285285
| "destructiveCommandGuardEnabled"
286+
| "alwaysDenyUnapprovedCommands"
286287
| "followupAutoApproveTimeoutMs"
287288
| "allowedCommands"
288289
| "deniedCommands"

‎scripts/stryker-diff.mjs‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,14 @@ export const PACKAGE_CONFIGS = [
5353
vitestConfig: "vitest.config.ts",
5454
vitestRelated: false,
5555
discoverRelatedTests: true,
56-
excludedPaths: ["src/esbuild.mjs", "src/eslint.config.mjs", "src/utils/vitest-verbosity.ts"],
56+
// CI build/coverage tooling under src/scripts is excluded like the other build scripts:
57+
// it is not shipped runtime code and is covered by src/scripts/__tests__/merge-lcov.spec.mjs.
58+
excludedPaths: [
59+
"src/esbuild.mjs",
60+
"src/eslint.config.mjs",
61+
"src/utils/vitest-verbosity.ts",
62+
"src/scripts/merge-lcov.mjs",
63+
],
5764
},
5865
]
5966

‎scripts/stryker-diff.test.mjs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,7 @@ describe("buildManifest", () => {
262262
{ status: "M", path: "webview-ui/src/value.visual.tsx" },
263263
{ status: "M", path: "webview-ui/src/main.tsx" },
264264
{ status: "M", path: "src/utils/vitest-verbosity.ts" },
265+
{ status: "M", path: "src/scripts/merge-lcov.mjs" },
265266
{ status: "M", path: "apps/cli/src/value.ts" },
266267
{ status: "M", path: "packages/cloud/src/types.ts" },
267268
],

0 commit comments

Comments
 (0)