Repository navigation
Read comma-separated subagent tool strings as separate tools - #925
Conversation
Claude documents a subagent's tools and disallowedTools as a YAML list or a comma-separated string such as "Read, Grep, Bash". toStringSlice turned any string into a one-element list, so "Read, Grep" became a single tool with a comma in its name. Claude re-emitted that entry, Factory and Gemini suppressed the subagent with subagent.factory.tool-unknown and subagent.gemini.tool-unknown, and a combined disallowedTools entry denied nothing. LoadSubagent now reads both fields with toToolList. A string is split on commas outside parentheses, so Agent(worker, researcher) stays whole, a backslash makes the next character literal, each entry is trimmed and blank entries are dropped. The loader is shared, so Claude and Cursor source layouts both change. YAML lists are read as before, and a single name is only trimmed. A string that names no tool, whose parentheses do not balance, or that ends in a lone backslash keeps the single entry earlier versions produced. Adyen#822 asks for such a value to be refused, but the v4 source-revision manifest lists every resource of the root, so refusing a subagent that earlier releases ingested changes the inventory and fails locks written by them with lock.manifest-mismatch, and the reverse. Factory and Gemini still refuse it at compile time. For subagents written in the string form, Marketplace capability entries (tool:<name>) and tool-grant finding texts now list each tool, and the content and archive digests of artifacts built from a re-ingested revision change. Source-revision digests hash raw source bytes and do not. Ingest-path cost on the same inputs, previous reader (which returned one wrong entry) then toToolList, as ns/op, B/op and allocs/op: 5-entry string 10 / 16 / 1 -> 66 / 80 / 1 scoped string 10 / 16 / 1 -> 68 / 64 / 1 malformed 10 / 16 / 1 -> 62 / 64 / 2 YAML list 26 / 80 / 1 -> 26 / 80 / 1 Refs Adyen#822. Co-Authored-By: Claude Fable 5.1 <[email protected]>
Co-Authored-By: Claude Fable 5.1 <[email protected]>
|
Warning Review limit reachedThis review ran on the open-source allowance, not this organization's plan, because the pull request author doesn't have an assigned seat. Waiting won't change this — ask an organization admin to assign them a seat, or add seats in Billing if every seat is already assigned, then retry. Next included review available in 51 minutes. View limit details
Comment |
There was a problem hiding this comment.
Code Review
This pull request implements splitting of comma-separated subagent tool strings into separate tools during ingestion for Claude and Cursor source layouts, while preserving backward compatibility for malformed strings. It includes comprehensive unit, benchmark, and end-to-end tests. Feedback on the changes suggests optimizing the benchmark in subagent_bench_test.go by moving the error validation check outside of the b.Loop() hot loop to prevent validation overhead from affecting benchmark accuracy.
| for b.Loop() { | ||
| var err error | ||
| if toolListBenchmarkResult, err = toToolList(tc.value); err != nil { | ||
| b.Fatal(err) | ||
| } | ||
| } |
There was a problem hiding this comment.
In Go benchmarks using b.Loop(), avoid performing validation assertions inside the hot loop to prevent validation overhead from affecting benchmark accuracy. Instead, declare result and error variables outside the loop, assign them within the loop, and perform validation checks after the loop has completed and stopped timing.
var err error
for b.Loop() {
toolListBenchmarkResult, err = toToolList(tc.value)
}
b.StopTimer()
if err != nil {
b.Fatal(err)
}References
- In Go benchmarks using 'b.Loop()', avoid performing validation assertions inside the hot loop to prevent validation overhead from affecting benchmark accuracy. Instead of validating before the loop and discarding results, declare result and error variables outside the loop, assign them within the loop, and perform validation checks after the loop has completed and stopped timing.
AlexanderGrooff
left a comment
There was a problem hiding this comment.
Nice work! LGTM
What changed and why
Claude Code documents a subagent's
toolsanddisallowedToolsas either a YAML list or a comma-separated string such asRead, Grep, Bash. The shared subagent loader kept any string as a single tool, sotools: Read, Grepreached the compilers as one tool namedRead, Grep: Claude re-emitted one combined entry, Factory and Gemini suppressed the subagent withsubagent.factory.tool-unknownandsubagent.gemini.tool-unknown, and a combineddisallowedToolsentry denied nothing.The loader now splits such a string on commas outside parentheses, so
Agent(worker, researcher), Readis two tools. A backslash makes the next character literal, each entry is trimmed, blank entries are dropped, and YAML lists are read as before.A value that is not a well-formed list (it names no tool, its parentheses do not balance, or it ends in a lone backslash) keeps the single entry earlier versions produced, so the source-revision inventory does not change and existing locks keep verifying. Claude and Cursor source layouts share the loader, so both change.
Refs #822 (the one-tool reading; the refusal of invalid values is discussed under Notes)
Type of change
claude,codex,cursor,devin,factory,gemini,opencode,windsurf)Harnesses affected
Ingestion: the
claudeandcursorsource layouts, and thedevinsource layout fordisallowedTools(Devin ingestion dropstools). Output: a subagent written in the string form in a Claude- or Cursor-layout source now produces exactly what the equivalent YAML list produces on all eight targets. Fortools: Read, Grepfrom a Claude-layout source that changes the output ofclaude,factory,geminiandopencode;codex,cursor,devinandwindsurfalready matched. A Cursor-layout source with the string form now converts to Claude instead of being suppressed withsubagent.claude.tool-unknown. A Devin-layout source withdisallowedTools: Bash, Writenow converts to Claude as two entries instead of one.How you tested it
Unit tests for the splitter (both keys, quoting, blanks, nested and escaped parentheses, escaped commas, dangling backslash, malformed strings, YAML lists), conversion tests from Claude, Cursor and Devin sources to the Claude, Factory, Gemini, Cursor and Devin targets, a benchmark, and two declarative e2e rows. The splitting tests fail on the parent commit. The malformed-string tests pass on the parent commit and fail against an implementation that refuses those values, which is the point of keeping them.
Before and after with the v5.0.3 release binary and this branch, installing a directory registry with
agents/helper.md(tools: Read, Grep, Glob) andagents/scoped.md(tools: Agent(worker, researcher), Read) for the Claude and Gemini harnesses, in an isolated HOME:Lock compatibility, in both directions. The registry is a Git repository with a valid skill and a subagent whose
toolsisRead, Bash(git diff(unbalanced parenthesis). The "refusing build" is an earlier revision of this change, built from the same tree, that rejected such a value at ingestion with a coded diagnostic:make validatepassesmake spec-checkpasses (needed when behavior or specs change)make validate-e2epasses (needed when sync, install, or harness output changes)make tui-qapasses (needed when the terminal UI changes)Specs and documentation
openspec/with a spec delta, or the commit carries aSpec-Impact: none - <reason>trailer because behavior is unchangedsite/docsThe change is archived as
2026-10-22-split-comma-separated-subagent-toolsand adds one requirement with five scenarios toharness-compilation. No page undersite/docsdescribes the subagenttoolsformat, so none changed;pkg/harness-compilation/CHANGELOG.mdrecords the change.Before you submit
LICENSENotes
tool:<name>) and the tool-grant finding text now list each tool, and the content and archive digests of artifacts built from a re-ingested revision change, because they are built from the ingested resource and now equal the YAML-list form. Payloads published earlier keep their bytes until the revision is published again. Source-revision digests hash raw source bytes and do not change." Read ", is now trimmed, so targets that suppressed it as an unknown tool now emit it.allowed-toolsgoes through the sametoStringSlicehelper and the skills page also documents space separation there, so it needs its own splitter. The Factory, Gemini and Devin ingesters keep privatetoStringSlicecopies with the same one-entry reading of a string. The Devin compiler splits tool entries on commas again and does not understand the grouping the loader now keeps. Cursor workspace subagents are absent from the source-revision inventory in v5.0.3 as well, Cursor workspace subagents are absent from the source-revision inventory in v5.0.3 as well, so a lock experiment with a Cursor-layout registry cannot exercise this change; the one above uses the Claude layout..sync --checkandsync --lockedfor the whole root withlock.manifest-mismatch, and the lock test above reproduces the mismatch with a build that refuses. Factory and Gemini still refuse the combined name at compile time. A refusal at ingestion would need a source-revision contract version. Whether to add it in this PR or in a separate change is for the maintainers to decide.