Skip to content

Read comma-separated subagent tool strings as separate tools - #925

Merged
AlexanderGrooff merged 2 commits into
Adyen:mainfrom
thalisantunes:fix/822-subagent-tools-string
Oct 23, 2026
Merged

AlexanderGrooff merged 2 commits into
Adyen:mainfrom
thalisantunes:fix/822-subagent-tools-string

Conversation

@thalisantunes

@thalisantunes thalisantunes commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What changed and why

Claude Code documents a subagent's tools and disallowedTools as either a YAML list or a comma-separated string such as Read, Grep, Bash. The shared subagent loader kept any string as a single tool, so tools: Read, Grep reached the compilers as one tool named Read, Grep: Claude re-emitted one combined entry, Factory and Gemini suppressed the subagent with subagent.factory.tool-unknown and subagent.gemini.tool-unknown, and a combined disallowedTools entry denied nothing.

The loader now splits such a string on commas outside parentheses, so Agent(worker, researcher), Read is two tools. A backslash makes the next character literal, each entry is trimmed, blank entries are dropped, and YAML lists are read as before.

A value that is not a well-formed list (it names no tool, its parentheses do not balance, or it ends in a lone backslash) keeps the single entry earlier versions produced, so the source-revision inventory does not change and existing locks keep verifying. Claude and Cursor source layouts share the loader, so both change.

Refs #822 (the one-tool reading; the refusal of invalid values is discussed under Notes)

Type of change

  • Bug fix
  • New feature or behavior change
  • Support for a harness (claude, codex, cursor, devin, factory, gemini, opencode, windsurf)
  • Documentation only
  • Build, CI, or tooling

Harnesses affected

Ingestion: the claude and cursor source layouts, and the devin source layout for disallowedTools (Devin ingestion drops tools). Output: a subagent written in the string form in a Claude- or Cursor-layout source now produces exactly what the equivalent YAML list produces on all eight targets. For tools: Read, Grep from a Claude-layout source that changes the output of claude, factory, gemini and opencode; codex, cursor, devin and windsurf already matched. A Cursor-layout source with the string form now converts to Claude instead of being suppressed with subagent.claude.tool-unknown. A Devin-layout source with disallowedTools: Bash, Write now converts to Claude as two entries instead of one.

How you tested it

Unit tests for the splitter (both keys, quoting, blanks, nested and escaped parentheses, escaped commas, dangling backslash, malformed strings, YAML lists), conversion tests from Claude, Cursor and Devin sources to the Claude, Factory, Gemini, Cursor and Devin targets, a benchmark, and two declarative e2e rows. The splitting tests fail on the parent commit. The malformed-string tests pass on the parent commit and fail against an implementation that refuses those values, which is the point of keeping them.

$ make validate
exit 0: tests for the changed packages, go vet, golangci-lint v2.14.0 "0 issues", build

$ make spec-check
Totals: 65 passed, 0 failed (65 items)
specaudit trace: ok (1172 scenarios, 1898 annotations)
specaudit archived: ok (2 active changes)

$ go run ./tools/specaudit gate -base <merge base> -head HEAD
specaudit gate: ok (2 behavior-bearing files with spec delta; no TUI-bearing changes)

$ go test ./pkg/harness-compilation/... -short -count=1
ok for all 33 packages

$ go -C e2e test -count=1 -run '^TestE2E$/^subagent/(comma_separated_tools_sync_as_separate_tools|malformed_tools_string_stays_one_entry)$' .
ok

$ make validate-e2e
ok  github.com/Adyen/agent-marketplace/e2e  144.015s
witness-check: 3 required tests ran and passed
shellcheck tools/harness-compat/*.sh: no findings (ShellCheck 0.11.0, run from its container image)

Before and after with the v5.0.3 release binary and this branch, installing a directory registry with agents/helper.md (tools: Read, Grep, Glob) and agents/scoped.md (tools: Agent(worker, researcher), Read) for the Claude and Gemini harnesses, in an isolated HOME:

v5.0.3
  .claude/agents/helper.md   tools: one entry, "Read, Grep, Glob"
  .gemini/agents/helper.md   absent, install exits 1 with subagent.gemini.tool-unknown
  .claude/agents/scoped.md   tools: one entry, "Agent(worker, researcher), Read"
this branch
  this branch
  .claude/agents/helper.md   tools: three entries, Read, Grep, Glob
  .gemini/agents/helper.md   tools: four entries, read_file, read_many_files, grep_search, glob
  .claude/agents/scoped.md   tools: two entries, Agent(worker, researcher) and Read
  .gemini/agents/helper.md   tools: read_file, read_many_files, grep_search, glob
  .claude/agents/scoped.md   tools: Agent(worker, researcher), Read
  .gemini/agents/scoped.md   absent: Gemini has no counterpart for the scoped Agent(...) tool,
                             so subagent.gemini.tool-unknown is reported, as for the YAML list form

Lock compatibility, in both directions. The registry is a Git repository with a valid skill and a subagent whose tools is Read, Bash(git diff (unbalanced parenthesis). The "refusing build" is an earlier revision of this change, built from the same tree, that rejected such a value at ingestion with a coded diagnostic:

v5.0.3 install of the skill writes a lock that lists the skill and the subagent
build that refused the value at ingestion:  sync --check exit 1 (lock.manifest-mismatch), sync --locked exit 1
this branch:                                sync --check exit 0, sync --locked exit 0
lock written by this branch, read by v5.0.3: sync --check exit 0, sync --locked exit 0
lock written by the refusing build, read by v5.0.3: sync --check exit 1 (lock.manifest-mismatch)
  • make validate passes
  • make spec-check passes (needed when behavior or specs change)
  • make validate-e2e passes (needed when sync, install, or harness output changes)
  • make tui-qa passes (needed when the terminal UI changes)

Specs and documentation

  • This change updates openspec/ with a spec delta, or the commit carries a Spec-Impact: none - <reason> trailer because behavior is unchanged
  • User-facing changes are reflected in site/docs

The change is archived as 2026-10-22-split-comma-separated-subagent-tools and adds one requirement with five scenarios to harness-compilation. No page under site/docs describes the subagent tools format, so none changed; pkg/harness-compilation/CHANGELOG.md records the change.

Before you submit

  • The change contains no secrets, tokens, internal hostnames, or machine-specific paths
  • I'm contributing this under the Apache 2.0 license, as described in LICENSE

Notes

  • Marketplace. For subagents written in the string form, the Marketplace capability entries (tool:<name>) and the tool-grant finding text now list each tool, and the content and archive digests of artifacts built from a re-ingested revision change, because they are built from the ingested resource and now equal the YAML-list form. Payloads published earlier keep their bytes until the revision is published again. Source-revision digests hash raw source bytes and do not change.
  • Trimming. A single name with surrounding whitespace, such as " Read ", is now trimmed, so targets that suppressed it as an unknown tool now emit it.
  • Deferred. Skill allowed-tools goes through the same toStringSlice helper and the skills page also documents space separation there, so it needs its own splitter. The Factory, Gemini and Devin ingesters keep private toStringSlice copies with the same one-entry reading of a string. The Devin compiler splits tool entries on commas again and does not understand the grouping the loader now keeps. Cursor workspace subagents are absent from the source-revision inventory in v5.0.3 as well, Cursor workspace subagents are absent from the source-revision inventory in v5.0.3 as well, so a lock experiment with a Cursor-layout registry cannot exercise this change; the one above uses the Claude layout..
  • Refusing invalid values (Claude subagent comma-separated tools string ingests as one tool #822). The issue also expects an unparseable value to be refused with a coded diagnostic. This PR does not refuse it. The v4 source-revision manifest lists every resource of a registry root, selected or not, so refusing at ingestion removes a subagent that earlier releases ingested from that inventory. A lock written by v5.0.3 then fails sync --check and sync --locked for the whole root with lock.manifest-mismatch, and the lock test above reproduces the mismatch with a build that refuses. Factory and Gemini still refuse the combined name at compile time. A refusal at ingestion would need a source-revision contract version. Whether to add it in this PR or in a separate change is for the maintainers to decide.

thalisantunes and others added 2 commits October 22, 2026 17:57
Claude documents a subagent's tools and disallowedTools as a YAML list or
a comma-separated string such as "Read, Grep, Bash". toStringSlice turned
any string into a one-element list, so "Read, Grep" became a single tool
with a comma in its name. Claude re-emitted that entry, Factory and Gemini
suppressed the subagent with subagent.factory.tool-unknown and
subagent.gemini.tool-unknown, and a combined disallowedTools entry denied
nothing.

LoadSubagent now reads both fields with toToolList. A string is split on
commas outside parentheses, so Agent(worker, researcher) stays whole, a
backslash makes the next character literal, each entry is trimmed and blank
entries are dropped. The loader is shared, so Claude and Cursor source
layouts both change. YAML lists are read as before, and a single name is
only trimmed.

A string that names no tool, whose parentheses do not balance, or that ends
in a lone backslash keeps the single entry earlier versions produced. Adyen#822
asks for such a value to be refused, but the v4 source-revision manifest
lists every resource of the root, so refusing a subagent that earlier
releases ingested changes the inventory and fails locks written by them
with lock.manifest-mismatch, and the reverse. Factory and Gemini still
refuse it at compile time.

For subagents written in the string form, Marketplace capability entries
(tool:<name>) and tool-grant finding texts now list each tool, and the
content and archive digests of artifacts built from a re-ingested revision
change. Source-revision digests hash raw source bytes and do not.

Ingest-path cost on the same inputs, previous reader (which returned one
wrong entry) then toToolList, as ns/op, B/op and allocs/op:

  5-entry string  10 / 16 / 1  ->  66 / 80 / 1
  scoped string   10 / 16 / 1  ->  68 / 64 / 1
  malformed       10 / 16 / 1  ->  62 / 64 / 2
  YAML list       26 / 80 / 1  ->  26 / 80 / 1

Refs Adyen#822.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
@coderabbitai

coderabbitai Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

This review ran on the open-source allowance, not this organization's plan, because the pull request author doesn't have an assigned seat. Waiting won't change this — ask an organization admin to assign them a seat, or add seats in Billing if every seat is already assigned, then retry.

Next included review available in 51 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
📥 Commits

Reviewing files that changed from the base of the PR and between 5b4d0e2 and b45f330.

📒 Files selected for processing: 13
  • e2e/table_test.go
  • openspec/changes/archive/2026-10-22-split-comma-separated-subagent-tools/.openspec.yaml
  • openspec/changes/archive/2026-10-22-split-comma-separated-subagent-tools/design.md
  • openspec/changes/archive/2026-10-22-split-comma-separated-subagent-tools/proposal.md
  • openspec/changes/archive/2026-10-22-split-comma-separated-subagent-tools/specs/harness-compilation/spec.md
  • openspec/changes/archive/2026-10-22-split-comma-separated-subagent-tools/tasks.md
  • openspec/specs/harness-compilation/spec.md
  • pkg/harness-compilation/CHANGELOG.md
  • pkg/harness-compilation/bundle/subagent.go
  • pkg/harness-compilation/bundle/subagent_bench_test.go
  • pkg/harness-compilation/bundle/subagent_test.go
  • pkg/harness-compilation/bundle/validate.go
  • pkg/harness-compilation/subagent_tool_string_test.go

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements splitting of comma-separated subagent tool strings into separate tools during ingestion for Claude and Cursor source layouts, while preserving backward compatibility for malformed strings. It includes comprehensive unit, benchmark, and end-to-end tests. Feedback on the changes suggests optimizing the benchmark in subagent_bench_test.go by moving the error validation check outside of the b.Loop() hot loop to prevent validation overhead from affecting benchmark accuracy.

Comment on lines +23 to +28
for b.Loop() {
var err error
if toolListBenchmarkResult, err = toToolList(tc.value); err != nil {
b.Fatal(err)
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In Go benchmarks using b.Loop(), avoid performing validation assertions inside the hot loop to prevent validation overhead from affecting benchmark accuracy. Instead, declare result and error variables outside the loop, assign them within the loop, and perform validation checks after the loop has completed and stopped timing.

			var err error
			for b.Loop() {
				toolListBenchmarkResult, err = toToolList(tc.value)
			}
			b.StopTimer()
			if err != nil {
				b.Fatal(err)
			}
References
  1. In Go benchmarks using 'b.Loop()', avoid performing validation assertions inside the hot loop to prevent validation overhead from affecting benchmark accuracy. Instead of validating before the loop and discarding results, declare result and error variables outside the loop, assign them within the loop, and perform validation checks after the loop has completed and stopped timing.

@AlexanderGrooff AlexanderGrooff left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! LGTM

@AlexanderGrooff
AlexanderGrooff added this pull request to the merge queue Oct 23, 2026
Merged via the queue into Adyen:main with commit c134877 Oct 23, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants