Skip to content

feat(tasks): warn when allowedTools match no reachable tool - #1607

Merged
mgoldsborough merged 2 commits into
NimbleBrainInc:mainfrom
Coding-Professional:feat/1601-task-tool-warnings
Oct 7, 2026
Merged

mgoldsborough merged 2 commits into
NimbleBrainInc:mainfrom
Coding-Professional:feat/1601-task-tool-warnings

Conversation

@Coding-Professional

@Coding-Professional Coding-Professional commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Creating a task with allowedTools: ["files__*", "mail__send"] previously saved silently even when mail__send was unavailable. Its first run then failed before any model call. Create and update now return an allowed_tool_unavailable warning for each unmatched entry, in both warnings and the response message. Saving remains allowed so a connector can be installed or granted later.

The warning uses the task owner's reachable tools in its workspace via Runtime.listToolsForWorkspace and the same unmatchedAllowedTools matcher as run-start validation. This keeps wildcard and granted my_ personal-connector behavior aligned with the actual run. Saved and inline tasks__run and tasks__run_batch calls receive the warning too. User docs and the generated schema declaration are updated.

Related issue

Closes #1601

Validation

  • bun test --no-env-file test/integration/task-allowed-tools.test.ts test/integration/task-assessment.test.ts test/integration/task-batches.test.ts — 26 passed, including create/update, saved and inline runs/batches, judge warnings, wildcard, and granted personal-connector cases.
  • bun run check and bun run check:web — passed.
  • Biome check of changed TypeScript files and git diff --check — passed.
  • cd docs && bun run build — passed, including internal-link validation.
  • bun run verify — attempted on native Windows; it stops at check:web-tests because that script expects an extensionless .bin/tsc despite the installed tsc.exe and a passing direct web typecheck. Tracked separately in Windows: TypeScript verification gates reject installed tsc.exe #1606; CI remains the full-suite signal.

Checklist

  • Linked the resolved issue with a closing keyword (Closes #N), if this PR fixes one.
  • Updated docs/ for a user-facing behavior change.
  • Ran bun run verify (or the relevant subset); Windows limitation documented above.

Copilot AI balanced review requested due to automatic review settings October 5, 2026 12:35
@Coding-Professional
Coding-Professional requested a review from a team as a code owner October 5, 2026 12:35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@mgoldsborough

Copy link
Copy Markdown
Contributor

QA Review: feat/1601-task-tool-warnings

Scope: 5 files, +155/−11 · worktree: .claude/worktrees/feat-1601-task-tool-warnings · reviewed: 855f07c0
Risk: LOW
Why this exists: a typo or renamed connector in a task's allowedTools now surfaces when the task is written, not as a failed first run (#1601).

Critical (must fix)

None.

Fix in-PR

None.

Suggestions (optional)

  • r1s1 [src/platform/tasks/source.ts:566-570] The catch { return []; } around listToolsForWorkspace is right not to fail the write, but it leaves no trace when discovery breaks, so a missing warning can't be told apart from a clean list. A one-line log.warn/debug with the workspace id would fix that. Optional.

What Looks Good

  • The warning uses the same reachable set (listToolsForWorkspace(ws, ownerId)) and the same matcher (unmatchedAllowedTools) as assertDeclaredToolsReachable at run start (src/runtime/runtime.ts:2186). That means wildcards, the RUN_DISCOVERY_TOOLS exemption, and granted my_ connectors give the same answer at write time and at run time. No second matcher can drift.
  • "Runs fail until…" is accurate for every task path: the run-start check is skipped only for trigger === "chat" (runtime.ts:1587), and a task run is never a chat run.
  • Judge and tool warnings resolve concurrently and merge through the existing withWarnings. The tasks UI renders w.message generically (TasksUI.tsx:296), so the new code needs no UI case.
  • Mutation-checked: with allowedToolWarnings replaced by [], 5 of the 13 tests in task-allowed-tools.test.ts fail (create, update, inline run, saved run, batches). The tests guard the behavior.

Body claims

  • ✅ The three named integration files give 26 passing tests (re-ran at 855f07c0). The wider test/unit/platform/tasks run plus those files gives 638 pass, 0 fail. bun run check is clean.
  • ✅ Uses the same matcher and reachable set as run-start validation (see above).
  • ✅ Covers saved and inline tasks__run / tasks__run_batch.
  • ✅ Generated tasks.d.ts matches the source union.

Verdict: ready to merge. Handoff: pr-adjudicate if the author wants to answer r1s1. Nothing blocks.

machine record
{
  "kind": "review", "pr": 1607, "round": 1,
  "reviewed": "855f07c030071010ffe7cdea90a020c5bf29f538", "delta_base": null,
  "risk": "LOW", "stop_gate": "n/a",
  "verdict": "Clean core; warning matches run-start validation exactly; tests mutation-verified. Ship.",
  "findings": [
    {"id": "r1s1", "bucket": "suggestion", "file": "src/platform/tasks/source.ts", "line": 568,
     "claim": "Tool-discovery failure in allowedToolWarnings is swallowed with no log, so a missing warning is indistinguishable from a clean list.",
     "evidence": "reproduced", "receipt": "read source.ts:563-579", "preconditions": null,
     "fix": "log the caught error at warn/debug before returning []", "scope": "peripheral"}
  ]
}

@mgoldsborough mgoldsborough added the qa-reviewed QA review completed with no critical issues label Oct 7, 2026
@mgoldsborough
mgoldsborough added this pull request to the merge queue Oct 7, 2026
@mgoldsborough

Copy link
Copy Markdown
Contributor

Thanks for the contribution @Coding-Professional

Merged via the queue into NimbleBrainInc:main with commit d627df6 Oct 7, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

qa-reviewed QA review completed with no critical issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Warn at task create/update when an allowedTools entry matches no available tool

3 participants