Repository navigation
fix(adkrest): render missing toolset tools and sub-agent connections in agent graph - #1730
karthik-0306 wants to merge 5 commits into
Conversation
4cff038 to
d0be297
Compare
jjsasha63
left a comment
There was a problem hiding this comment.
The root-level case from #1726 works: I removed each half of teh fix in turn and TestAgentGraphWithToolsetAndSubAgent went red both times. Outside that case,the new sub-agent edge breaks the graph for any coordinator with a workflow sub-agent, and resolving toolsets lets a single toolset fail the whole graph.
I ran probe tests against GetAgentGraph and the build_graph_image handler and compared them with main at 49904c0:
- An llmagent root with a sequential, loop or parallel sub-agent gets the DOT body
error: pipeline is not a node or a subgraph, served with HTTP 200. Main renders it. - A toolset whose
Toolsreadsctx.UserID(), or atool.FilterToolsetpredicate that readsctx.ReadonlyState(), panics with a nil pointer dereference and the client gets EOF. main renders it. - An mcptoolset with an unreachable server, or with per-user
auth/gcpauth, turns the whole graph into a 500. Main renders it. - An llmagent inside a sequential agent gets no edges to its toolset tools or sub-agents.
Parity with adk-python 2.11.0 (src/google/adk/cli/agent_graph.py, src/google/adk/agents/llm_agent.py): the root-level behavior matches agent_graph.py:289-301. Python skips the edge when the sub-agent is a workflow agent and conects the parent into the cluster instead (agent_graph.py:159-160, 177-178, 196-199, 292-297). It also logs and skips a toolset that fails to load, so the rest of the graph still renders (llm_agent.py:224-245).
…in agent graph Fixes google#1726
d0be297 to
290c5dc
Compare
Thanks for the thorough review and edge-case testing. I've updated the PR to address all of these issues to bring it into full parity with the Python SDK:
|
karolpiotrowicz
left a comment
There was a problem hiding this comment.
Two things need to change before this merges. Toolset tool names go into the DOT source unquoted, so an MCP toolset with a hyphenated tool name now makes the whole graph unrenderable. The new failure paths also have no tests.
The four cases from the earlier review render correctly on this head. I drove each one through BuildGraphImageHandler and GetAgentGraph. An llmagent with a sequential, loop or parallel sub-agent gets an edge into the cluster with lhead. Toolsets that panic on ctx.UserID(), filter on ReadonlyState() or return an error are skipped and the rest of the graph renders. An llmagent inside a sequential agent gets edges to its tools, its toolset tools and its sub-agent. TestAgentGraphWithToolsetAndSubAgent fails on the PR's base (49904c0) and passes on this head (290c5dc).
Must change before merge:
-
Toolset tool names reach the DOT output as bare IDs. The new loop at agentgraphgenerator.go#L394-L402 hands each toolset tool to
drawNodeanddrawEdge, which use the name as the node ID without quoting (L321, L343). MCP servers pick these names, and hyphens are common (Context7'sresolve-library-id, for example). For an llmagent with one direct tool and a toolset returningresolve-library-id, the generated DOT containsroot->resolve-library-id[...], and Graphviz rejects it withsyntax error in line 5 near '-'. The web UI then has nothing it can draw. Before this change the same agent rendered, just without the toolset tool. A name containing"fails the same way. A directly configured tool with a hyphen already fails like this on main, so the gap is older than this PR, but the PR now sends every MCP tool name through it. Whatever the fix, every node ID the generator emits has to be valid DOT for any tool name, on edges as well as nodes. -
The new failure paths have no tests. Under the package's tests, the recover block at L381-L383 and the skip branch at L388-L392 never run, so deleting either one leaves the suite green. No test builds an llmagent with a workflow sub-agent either, which is the
drawClusterEdgecall at L411-L416, andTestAgentGraphWithToolsetAndSubAgentis the only test that usesToolsetsat all, so a toolset on an llmagent nested inside a workflow agent is not covered. The cases worth pinning are an erroring toolset, a panicking toolset, an llmagent with a sequential sub-agent, and a toolset on an llmagent inside a sequential agent. Parsing the output with the existingparseGraphhelper instead ofstrings.Containswould also catch the ID problem above:gographviz.ParseStringrejects the hyphenated DOT.
Reproduction for the hyphenated name (drop into server/adkrest/internal/services, run go test -run TestHyphenatedToolsetName -v ./server/adkrest/internal/services/, then feed the logged DOT to dot -Tsvg)
package services
import (
"context"
"testing"
"github.com/awalterschulze/gographviz"
"google.golang.org/adk/v2/agent"
"google.golang.org/adk/v2/agent/llmagent"
"google.golang.org/adk/v2/tool"
)
type namedTool struct{ n string }
func (t namedTool) Name() string { return t.n }
func (t namedTool) Description() string { return "" }
func (t namedTool) IsLongRunning() bool { return false }
type mcpLikeToolset struct{}
func (mcpLikeToolset) Name() string { return "mcp" }
func (mcpLikeToolset) Tools(agent.ReadonlyContext) ([]tool.Tool, error) {
return []tool.Tool{namedTool{"resolve-library-id"}}, nil
}
func TestHyphenatedToolsetName(t *testing.T) {
root, err := llmagent.New(llmagent.Config{Name: "root", Model: &dummyLLM{}, Toolsets: []tool.Toolset{mcpLikeToolset{}}})
if err != nil {
t.Fatal(err)
}
dot, err := GetAgentGraph(context.Background(), root, nil)
if err != nil {
t.Fatal(err)
}
t.Log(dot)
if _, err := gographviz.ParseString(dot); err != nil {
t.Errorf("generated DOT does not parse: %v", err)
}
}…n agent graph Address review feedback on the agent graph changes for google#1726. Quote DOT IDs that are not plain identifiers. Toolset tools now reach the graph, and MCP servers pick their own tool names, so a name like Context7's resolve-library-id made Graphviz reject the whole source. A hyphenated direct tool or agent already failed this way on main, and a bare `node`, `ns:tool` or a name with a double quote misrendered silently. Every node, edge endpoint, cluster and ltail/lhead value now goes through dotID: plain ASCII identifiers that are not DOT keywords stay bare, so graphs that rendered before produce the same source, and anything else is quoted with backslash and double quote escaped. Labels get the same escaping; highlighting still matches raw names. Give toolset resolution the agent being drawn, so a toolset or a tool.FilterToolset predicate calling ctx.AgentName() works instead of panicking and dropping the toolset. There is still no session or user, so a toolset that needs them is skipped, as adk-python does. Log a skipped toolset at WARN with the agent, toolset name, its Go type, and the type of the error or panic value. The values are not logged, since they can carry a server URL or credentials. Skip nil toolsets before logging. Bound toolset resolution with one 5-second budget for the whole walk. An MCP server that accepts the connection and never answers held the graph for over 90 seconds in a local probe. 10 seconds, as exp/appinfo uses, plus the MCP SDK's ~5 seconds of session cleanup reached the web launcher's default 15-second write timeout and the client got no response; with 5 seconds it answers 200 in about 10 seconds. Add tests that assert on the parsed graph for an erroring and a panicking toolset, an llmagent with a sequential, loop or parallel sub-agent, an llmagent with tools, a toolset and a sub-agent inside a sequential agent, names that are not DOT identifiers (also through BuildGraphImageHandler), the agent in the toolset context, the log records, and the resolution budget. Related: google#1726
47489ed to
c0ab228
Compare
|
@karolpiotrowicz @jjsasha63 thanks for the reviews. Pushed 47489ed, which addresses everything:
Full suite with |
Every item this review raised is fixed and verified on the current head.
karolpiotrowicz
left a comment
There was a problem hiding this comment.
The latest commit (c0ab228) adds two problems that need fixing before this PR merges. Building the graph can now leave a stdio MCP toolset broken for every later run of the agent. The comment that justifies the new 5s budget for building the graph also does not hold for stdio servers.
Both points from my earlier review are fixed. I rendered 40 names that are not plain identifiers as toolset tools, direct tools, sub-agents and workflow agents. They cover hyphens, quotes, trailing backslashes, DOT keywords, colons as in ns:tool, newlines, the empty string and invalid UTF-8. An agent cannot have an empty name, so the 40 names in four positions make 158 graphs rather than 160. On this head Graphviz accepts all of them. The 154 that are valid UTF-8 have the expected nodes and edges. On the previous head (290c5dc), 129 of the 158 failed to parse or had the wrong nodes and edges. The failure paths have tests now, and a tree that uses only plain names gives byte-identical DOT to the previous head. The checks I listed last time all pass again.
Must change before merge:
-
A graph request that runs out of budget while a stdio MCP server is starting breaks its toolset until the ADK server restarts. The 5s budget from agentgraphgenerator.go#L547-L549 at c0ab228 sets the deadline on the context
mcptoolsetconnects with (client.go#L174-L194). When the MCP server needs longer than 5s to answerinitialize, the MCP SDK closes the session and kills the server's process (go-sdk v1.8.0 client.go#L390-L393).mcptoolsetkeeps the sameCommandTransport, whoseexec.Cmdcannot be started a second time (go-sdk v1.8.0 cmd.go#L29-L46). Every later run of any agent using that toolset then fails withfailed to extract tools from the tool set "mcp_tool_set": ... exec: Stdout already set(tools_processor.go#L40-L44).The toolset breaks before anyone runs the agent, because the dev UI requests the graph as soon as an app is selected.
CommandTransportis the setup themcptoolsetdocs show (set.go#L63-L65). A server launched throughnpx -yoruvxdownloads its package before it starts, so its firstinitializewaits on the network. The graph build connects each toolset in turn under one 5s budget. A server that starts after another one gets what is left.On the previous head the reproduction below passes, because the graph build waits for
initialize, and later agent runs reuse the MCP session that the graph build opened. The underlying problem is older than this PR.mcptoolsetreuses aCommandTransportwhose process was already killed, and on main a first agent run that is cancelled while the server starts breaks the toolset the same way. The new budget breaks the toolset this way whenever a server needs more than 5s to start, without anyone cancelling anything. Any fix has to leave later runs unaffected by building the graph. -
The budget comment's timing is wrong for stdio servers. agentgraphgenerator.go#L518-L522 says an MCP toolset spends about 5s more after its context ends, which keeps the response under the launcher's 15s write timeout. For a stdio server the SDK waits 5s after closing stdin and another 5s after SIGTERM before it kills the process (go-sdk v1.8.0 cmd.go#L69-L107). The 5s budget plus up to 10s of shutdown reaches the 15s write timeout. In my test the server never answers and ignores SIGTERM. A handler calling
GetAgentGraphWithThemeran behind anhttp.Serverwith the launcher's 15sWriteTimeout, and after 15s the client gotEOFwith no body.
Reproduction for the stdio toolset (save as server/adkrest/internal/services/stdio_graph_repro_test.go and run go test -run TestAgentRunAfterGraphWithSlowStdioServer ./server/adkrest/internal/services/. It fails on this head and passes on the previous head.)
package services
import (
"context"
"iter"
"os"
"os/exec"
"testing"
"time"
"github.com/modelcontextprotocol/go-sdk/mcp"
"google.golang.org/genai"
"google.golang.org/adk/v2/agent"
"google.golang.org/adk/v2/agent/llmagent"
"google.golang.org/adk/v2/model"
"google.golang.org/adk/v2/runner"
"google.golang.org/adk/v2/session"
"google.golang.org/adk/v2/tool"
"google.golang.org/adk/v2/tool/mcptoolset"
)
// Re-executed as a stdio MCP server that needs 7s before it answers.
func TestSlowStdioServerHelper(t *testing.T) {
if os.Getenv("SLOW_MCP_SERVER") != "1" {
t.Skip("helper process")
}
time.Sleep(7 * time.Second)
srv := mcp.NewServer(&mcp.Implementation{Name: "slow", Version: "v0"}, nil)
mcp.AddTool(srv, &mcp.Tool{Name: "lookup"}, func(context.Context, *mcp.CallToolRequest, struct{}) (*mcp.CallToolResult, struct{}, error) {
return &mcp.CallToolResult{}, struct{}{}, nil
})
_ = srv.Run(context.Background(), &mcp.StdioTransport{})
os.Exit(0)
}
type okModel struct{}
func (okModel) Name() string { return "ok" }
func (okModel) GenerateContent(context.Context, *model.LLMRequest, bool) iter.Seq2[*model.LLMResponse, error] {
return func(yield func(*model.LLMResponse, error) bool) {
yield(&model.LLMResponse{Content: genai.NewContentFromText("ok", genai.RoleModel)}, nil)
}
}
func TestAgentRunAfterGraphWithSlowStdioServer(t *testing.T) {
cmd := exec.Command(os.Args[0], "-test.run=^TestSlowStdioServerHelper$")
cmd.Env = append(os.Environ(), "SLOW_MCP_SERVER=1")
ts, err := mcptoolset.New(mcptoolset.Config{Transport: &mcp.CommandTransport{Command: cmd}})
if err != nil {
t.Fatal(err)
}
root, err := llmagent.New(llmagent.Config{Name: "root", Model: okModel{}, Toolsets: []tool.Toolset{ts}})
if err != nil {
t.Fatal(err)
}
// The dev UI requests the graph when the app is opened, before any run.
if _, err := GetAgentGraph(context.Background(), root, nil); err != nil {
t.Fatal(err)
}
r, err := runner.New(runner.Config{AppName: "app", Agent: root, SessionService: session.InMemoryService(), AutoCreateSession: true})
if err != nil {
t.Fatal(err)
}
for _, err := range r.Run(context.Background(), "user", "s1", genai.NewContentFromText("hi", genai.RoleUser), agent.RunConfig{}) {
if err != nil {
t.Fatalf("agent run after the graph: %v", err)
}
}
}On your logging questions
WARN fits the level table in AGENTS.md (AGENTS.md#L163-L169), since the graph still renders. Logging the error's type rather than its text follows the AGENTS.md rule against URLs in log lines (AGENTS.md#L134-L143), because an MCP error can carry the server's URL. The //nolint:forbidigo follows the same pattern as exp/appinfo, which also skips a toolset that cannot list its tools and logs only the error's type (appinfo.go#L309).
Building the graph listed each toolset under a 5-second deadline. When a stdio MCP server took longer than that to answer initialize, as one launched through npx -y or uvx does while it downloads, the deadline made the MCP SDK close the session and kill the server's process. mcptoolset keeps the same CommandTransport, which cannot start the process again, so every later run of the agent failed with "exec: Stdout already set". The dev UI requests the graph as soon as an app is opened, so this happened before anyone ran the agent. The deadline also did not bound the response: the SDK waits up to 10 seconds more for a stdio server to exit, which reached the web launcher's 15-second write timeout. List each toolset on its own goroutine under a context that keeps the request's values but not its cancellation, and only stop waiting once the budget is spent. A listing that has not returned is left out of that graph, logged with kind "timeout", and finishes in the background, so later runs reuse the session it opened. Start every toolset in the tree when the graph starts, so each gets the whole budget, and let a graph built while a listing is still running wait for that one, so a server that never answers leaves one goroutine behind instead of one per request. List toolsets with a context that reads as empty instead of panicking for the user, session, state and invocation a run would supply, as exp/appinfo does, so toolsets such as tool/toolsearch that read state list their tools. It reports no invocation identity, so nothing lists tools as whoever requested the graph. Read a skipped toolset's name for the log under a recover: a wrapper around a nil toolset, such as tool.WithConfirmation(nil, ...), passes the nil check and panics in Name as well as Tools. Related: google#1726
|
@karolpiotrowicz thanks, your stdio repro was spot on. Both must-fix points are fixed in 07f79b4. I also merged main first, since mcptoolset changed in #1755 and #1844. 1. The graph broke stdio toolsets for later runs The graph no longer cancels a toolset's listing; it only stops waiting for it.
Your End to end with the web launcher and a
2. The budget comment's timing Nothing is canceled now, so no MCP shutdown time is added to the response, and the comment is rewritten to say so. With a stdio server that never answers and ignores SIGTERM, behind the launcher's 15s write timeout:
Non-blocking
Tests I broke each new piece of code in turn and saw the matching test fail:
Earlier mutations, such as quoting, the skip branch and the sub-agent and nested edges, still fail their tests. On a clean checkout with Go 1.26.10:
Older problems, out of scope I can open issues for these:
|
karolpiotrowicz
left a comment
There was a problem hiding this comment.
My change request from my earlier review stands. A graph request can still leave an MCP toolset unusable for every later run and block its Close, which is what I asked the fix to rule out.
What the new commit fixes:
- With a stdio server that needs 7s before it answers, the graph now returns in 5s without that server's tools. Two agent runs started after the graph request now succeed, and the next graph request shows the server's tool. On the previous head (c0ab228) both runs failed with
exec: Stdout already set. - The graph's response now stays within its 5s budget. With a stdio server that never answers and ignores SIGTERM, a graph endpoint served under the launcher's 15s write timeout returns 200 in 5s.
- The toolset name for the log is now read under a
recover. toolsearchnow lists its own tools in the graph.
The build, lint and test checks from my earlier reviews pass again on the current head, 07f79b4, and every line link below is pinned to it.
On the older problems you listed, #2111 already covers mcptoolset reusing a CommandTransport whose process was killed, and the exp/appinfo timeout is worth an issue of its own.
Must change before merge
A graph request against an MCP server that never answers leaves that toolset locked for the life of the process. The listing runs under context.WithoutCancel (agentgraphtoolsets.go#L111), so nothing can end it. While mcptoolset connects, getSession holds the connection's lock for the whole connect (client.go#L211-L231, client.go#L264-L272). The MCP SDK puts no timeout of its own on connecting. Against a server that accepts the connection and never answers, holding the lock through a connect that nothing can cancel has two effects:
- Later runs never reconnect. Every later agent run waits for the lock until its own deadline and fails. For an HTTP server, later runs keep failing after the server recovers, because no new connect is ever tried. In the reproduction below the server holds its first request and serves every later one normally. After a graph request, two runs with a 6s deadline both fail, and the server sees one request in total. Without the graph request the second run succeeds. On the previous head the graph's deadline released the lock and both runs succeeded. With
Config.Authset toauth.StaticToken(...), which uses the per-credential session pool, both runs fail the same way on this head. Closenever returns. It takes the same lock and has no context (client.go#L91-L95), so it blocks for good. A program that defersClosethen cannot shut down, and deferring it is what themcptoolsetdocs show (set.go#L51-L69). A stdio server that never answers is never stopped either.Closeblocks and the server stays alive whether or not the server ignores SIGTERM. With a server that keeps default signal handling, a goroutine dump showsClosewaiting while the listing sits inClient.Connect. Without the graph request,Closereturns at once. It also returns at once on the merge base with main (be8b100) and on the previous head, with or without a graph request.
The comment at agentgraphtoolsets.go#L40-L47 says that a listing left running finishes in the background and that later agent runs reuse its session. That does not hold when the server never answers. Whatever the fix, building the graph must not leave a toolset in a state that breaks or blocks a later run, a later graph or Close, including when its server never answers.
On the previous head, ending the listing at a deadline killed a stdio server's process. The toolset then stayed broken, because mcptoolset keeps reusing a transport whose process is gone (#2111). So a deadline that frees the lock within seconds brings back the failure from my earlier review for any server slower than that deadline.
Reproduction (save as server/adkrest/internal/services/stalled_server_repro_test.go and run go test -run TestAgentRunAfterGraphWithStalledServer ./server/adkrest/internal/services/. It fails on this head and passes on the previous head.)
package services
import (
"context"
"io"
"iter"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
"github.com/modelcontextprotocol/go-sdk/mcp"
"google.golang.org/genai"
"google.golang.org/adk/v2/agent"
"google.golang.org/adk/v2/agent/llmagent"
"google.golang.org/adk/v2/model"
"google.golang.org/adk/v2/runner"
"google.golang.org/adk/v2/session"
"google.golang.org/adk/v2/tool"
"google.golang.org/adk/v2/tool/mcptoolset"
)
type replyModel struct{}
func (replyModel) Name() string { return "reply" }
func (replyModel) GenerateContent(context.Context, *model.LLMRequest, bool) iter.Seq2[*model.LLMResponse, error] {
return func(yield func(*model.LLMResponse, error) bool) {
yield(&model.LLMResponse{Content: genai.NewContentFromText("ok", genai.RoleModel)}, nil)
}
}
// The MCP server holds its first request until the test ends and serves every
// later request normally, like a server that stalls once and recovers.
func TestAgentRunAfterGraphWithStalledServer(t *testing.T) {
srv := mcp.NewServer(&mcp.Implementation{Name: "stall", Version: "v0"}, nil)
mcp.AddTool(srv, &mcp.Tool{Name: "lookup"}, func(context.Context, *mcp.CallToolRequest, struct{}) (*mcp.CallToolResult, struct{}, error) {
return &mcp.CallToolResult{}, struct{}{}, nil
})
h := mcp.NewStreamableHTTPHandler(func(*http.Request) *mcp.Server { return srv }, nil)
stop := make(chan struct{})
var requests atomic.Int32
hs := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if requests.Add(1) == 1 {
select {
case <-stop:
case <-r.Context().Done():
}
return
}
h.ServeHTTP(w, r)
}))
t.Cleanup(func() { close(stop); hs.CloseClientConnections(); hs.Close() })
ts, err := mcptoolset.New(mcptoolset.Config{Endpoint: hs.URL})
if err != nil {
t.Fatal(err)
}
root, err := llmagent.New(llmagent.Config{Name: "root", Model: replyModel{}, Toolsets: []tool.Toolset{ts}})
if err != nil {
t.Fatal(err)
}
// Remove this call and the second run succeeds.
if _, err := GetAgentGraph(context.Background(), root, nil); err != nil {
t.Fatal(err)
}
r, err := runner.New(runner.Config{AppName: "app", Agent: root, SessionService: session.InMemoryService(), AutoCreateSession: true})
if err != nil {
t.Fatal(err)
}
var runErr error
for _, sid := range []string{"s1", "s2"} {
ctx, cancel := context.WithTimeout(context.Background(), 6*time.Second)
runErr = nil
for _, err := range r.Run(ctx, "user", sid, genai.NewContentFromText("hi", genai.RoleUser), agent.RunConfig{}) {
if err != nil {
runErr = err
break
}
}
cancel()
}
if runErr != nil {
t.Errorf("second run, after the server recovered: %v (server saw %d requests)", runErr, requests.Load())
}
closed := make(chan struct{})
go func() { _ = ts.(io.Closer).Close(); close(closed) }()
select {
case <-closed:
case <-time.After(10 * time.Second):
t.Error("Close still blocked after 10s")
}
}| if llmAgent, ok := root.(llmagentinternal.Agent); ok { | ||
| for _, toolset := range llmagentinternal.Reveal(llmAgent).Toolsets { | ||
| use := toolsetUse{agent: root, toolset: toolset} | ||
| if toolset == nil || !use.keyable() { |
There was a problem hiding this comment.
toolsetLister.start skips any toolset whose type cannot be a map key. A value type with a slice field is one example, like catalog in the toolsearch example (main.go#L107-L112 at the current head, 07f79b4). tools then calls listTools for such a toolset only when the walk reaches it. That listing is not recorded in running, so each later graph request starts its own.
Skipping such a toolset has two effects on the current head, 07f79b4. First, when an earlier toolset is still listing as the budget runs out, a toolset that cannot be a map key gets none of the budget. That toolset is then left out of each graph built while the earlier one is still listing, even when that toolset's listing is instant. Second, if a toolset that cannot be a map key hangs in Tools, each graph request leaves one more blocked call behind, whereas a pointer toolset leaves one in total.
The toolsetWaitBudget comment says every toolset starts listing at once and gets the whole budget, which does not hold for these toolsets. ADK's own toolset constructors all return pointers, so the skipped start need not block the merge.
Test showing both (save as server/adkrest/internal/services/unkeyable_toolset_repro_test.go and run go test -run TestToolsetThatCannotBeAMapKey ./server/adkrest/internal/services/. It fails on this head and passes when sliceToolset is passed as a pointer.)
package services
import (
"context"
"strings"
"sync/atomic"
"testing"
"google.golang.org/adk/v2/agent"
"google.golang.org/adk/v2/agent/llmagent"
"google.golang.org/adk/v2/tool"
)
// blockedToolset can be a map key. Its Tools blocks until release is closed.
type blockedToolset struct{ release chan struct{} }
func (blockedToolset) Name() string { return "blocked" }
func (b blockedToolset) Tools(agent.ReadonlyContext) ([]tool.Tool, error) {
<-b.release
return nil, nil
}
// sliceToolset cannot be a map key. Its Tools blocks while release is open.
type sliceToolset struct {
names []string
release chan struct{}
calls *atomic.Int32
}
func (sliceToolset) Name() string { return "slice" }
func (s sliceToolset) Tools(agent.ReadonlyContext) ([]tool.Tool, error) {
if s.calls != nil {
s.calls.Add(1)
}
if s.release != nil {
<-s.release
}
return []tool.Tool{newTestTool("slice_tool")}, nil
}
func TestToolsetThatCannotBeAMapKey(t *testing.T) {
release := make(chan struct{})
t.Cleanup(func() { close(release) })
// Listed instantly, but behind a toolset that is still listing.
root, err := llmagent.New(llmagent.Config{Name: "root", Model: &dummyLLM{},
Toolsets: []tool.Toolset{blockedToolset{release}, sliceToolset{names: []string{"a"}}}})
if err != nil {
t.Fatal(err)
}
if dot, _ := GetAgentGraph(context.Background(), root, nil); !strings.Contains(dot, "slice_tool") {
t.Error("slice_tool is left out of the graph")
}
// Hangs: three graph requests. Each call to Tools runs on a goroutine of
// its own and stays blocked.
var calls atomic.Int32
hung, err := llmagent.New(llmagent.Config{Name: "hung", Model: &dummyLLM{},
Toolsets: []tool.Toolset{sliceToolset{names: []string{"a"}, release: release, calls: &calls}}})
if err != nil {
t.Fatal(err)
}
for range 3 {
_, _ = GetAgentGraph(context.Background(), hung, nil)
}
if n := calls.Load(); n > 1 {
t.Errorf("3 graph requests left %d blocked calls to Tools, want 1 shared", n)
}
}963c48e to
07f79b4
Compare
|
@karolpiotrowicz thanks, you're right: on 07f79b4 a server that never answers keeps the toolset locked, so later runs fail and
Which would you prefer? Happy to go either way. |
Linked issue
Closes: #1726
2. Describe the change:
Problem: The agent graph generator leaves out tools from Toolsets (like MCP, skills) because
buildGraphonly reads the explicitagent.Toolsfield. It also draws sub-agents of anllmagentwithout edges to their parent, so they appear as disconnected nodes.Solution:
kind=timeout. A listing is never canceled: canceling a stdio MCP toolset while it connects kills the server's process, and mcptoolset can't start it again, so every later run of the agent would fail. An unfinished listing completes in the background, and later runs reuse the session it opened. A listing still running is shared by later graphs, so a server that never answers leaves one goroutine behind, not one per request.tool/toolsearchthat read state still list their tools. It reports no invocation identity, so nothing lists tools as whoever requested the graph.lhead, as in adk-python'sagent_graph.py. Edges of an llmagent nested in a workflow cluster are drawn on the graph that gets serialized.ltail/lheadvalues that aren't plain identifiers are quoted and escaped. MCP tool names likeresolve-library-idpreviously made Graphviz reject the whole graph.Behavior change
Toolset.Tools, so an MCP toolset connects to its server. The graph waits at most 5s. A toolset that fails or isn't ready by then is left out and logged at WARN; the client gets no error. Listings are never canceled, so later agent runs are unaffected, and a server that was still starting appears on the next render.Testing Plan
Unit Tests: On a clean checkout with Go 1.26.10,
go test -race -mod=readonly -count=1 -shuffle=on workpasses.go mod tidy -diff,go build,go vetandgolangci-lint run(v2.3.1) are clean in both modules. The two touched packages also pass 10 repeated shuffled-raceruns.With your source change reverted and your tests kept, which test fails?
I removed each piece of the change separately and confirmed the matching test fails:
TestAgentGraphWithToolsetAndSubAgent,TestAgentGraphLLMAgentWithWorkflowSubAgentTestAgentGraphSkipsToolsetsThatCannotListTools(panic/error)TestAgentGraphLLMAgentInsideWorkflowTestDotID,TestAgentGraphQuotesNamesThatAreNotDOTIdentifiers,TestBuildGraphImageHandlerQuotesToolsetToolNamesTestAgentGraphToolsetSeesItsAgentName()panics →TestAgentGraphLogsSkippedToolsetsTestAgentGraphLeavesAStartingToolsetUsable,TestAgentRunAfterGraphWithSlowStdioServer(a real stdio MCP server throughCommandTransport)TestAgentGraphDoesNotWaitPastItsBudgetTestAgentGraphListsToolsetsTogetherTestAgentGraphSharesAListingStillRunningTestAgentGraphListsToolsetsWithoutASession(usestool/toolsearch)Manual End-to-End (E2E) Tests:
I ran the reproduction from #1726 with
web -port 8097 api -include_debug_apiand requestedcurl /api/dev/apps/root/build_graph_image.toolset_toolwas missing andhelperwas a floating node.rootconnects todirect_tool,toolset_toolandhelper.resolve-library-idandsay"hirender.CommandTransport):POST /runafterwards returns 200, and the next graph shows the server's tool.exec: Stdout already set.The graph outputs render with Graphviz 16.1.0.
Checklist