Add bundle MCP diagnostics timeline spans by LLagoon3 · Pull Request #81787 · openclaw/openclaw

LLagoon3 · 2026-05-14T10:25:01Z

Summary

Problem: bundle-tools could take noticeable time during embedded agent prep, but diagnostics only exposed a single opaque timing bucket.
Why it matters: when configured/bundled MCP tools are slow to discover, maintainers cannot tell which server or MCP operation is responsible.
What changed: added diagnostics timeline spans around each bundle MCP server startup, connection, and tools/list catalog discovery.
What did NOT change (scope boundary): no MCP protocol behavior, tool naming/materialization, config shape, or user-facing defaults changed.

Change Type (select all)

Scope (select all touched areas)

Linked Issue/PR

Closes observability: emit per-MCP-server sub-spans inside bundle-tools so cold-start cost is attributable #81595
This PR fixes a bug or regression

Real behavior proof (required for external PRs)

Behavior or issue addressed: bundle MCP discovery now emits per-server diagnostics timeline spans instead of only contributing to the opaque bundle-tools prep stage.
Real environment tested: local OpenClaw checkout on Linux, running the real MCP runtime against a stdio MCP probe server.
Exact steps or command run after this patch:

OPENCLAW_VITEST_INCLUDE_FILE=/tmp/oc81595-vitest-include.json \
OPENCLAW_VITEST_MAX_WORKERS=1 \
OPENCLAW_TEST_PROJECTS_SERIAL=1 \
node scripts/run-vitest.mjs run --config test/vitest/vitest.agents.config.ts --reporter dot --maxWorkers=1

Evidence after fix:

Test Files  1 passed (1)
Tests       12 passed (12)

The new coverage starts a real stdio MCP probe server, materializes bundle MCP tools, and reads the diagnostics timeline JSONL. It asserts span.end events for:

bundle-mcp.server
bundle-mcp.connect
bundle-mcp.tools-list

Observed result after fix: the timeline includes the expected bundle MCP server/connect/tools-list spans with durations.
What was not tested: full repository lint/check/test was intentionally not run on this machine because those large suites can create high memory pressure.
Before evidence (optional but encouraged): issue observability: emit per-MCP-server sub-spans inside bundle-tools so cold-start cost is attributable #81595 has a reproduction comment showing bundle-tools:1680ms@2047ms with no per-MCP-server/tools-list breakdown.

Root Cause (if applicable)

Root cause: bundle MCP catalog discovery connected to each configured server and listed its tools inside one larger prep stage without emitting nested diagnostics timeline spans.
Missing detection / guardrail: no test asserted timeline observability for bundle MCP server catalog discovery.
Contributing context (if known): plugin tool factories already have detailed timings, so the equivalent bundle MCP path stood out as a black box.

Regression Test Plan (if applicable)

Coverage level that should have caught this:
- Unit test
- Seam / integration test
- End-to-end test
- Existing coverage already sufficient
Target test or file: src/agents/pi-bundle-mcp-runtime.test.ts
Scenario the test should lock in: materializing tools from a real stdio MCP server with diagnostics timeline enabled writes bundle-mcp.server, bundle-mcp.connect, and bundle-mcp.tools-list spans.
Why this is the smallest reliable guardrail: it exercises the actual bundle MCP runtime and MCP client path without requiring a full agent/model run.
Existing test that already covers this (if any): none.
If no new test is added, why not: N/A.

User-visible / Behavior Changes

None. This only adds diagnostics timeline events when diagnostics timeline output is enabled.

Diagram (if applicable)

Before:
agent prep -> bundle-tools timing bucket -> no per-MCP-server detail

After:
agent prep -> bundle-tools timing bucket
           -> bundle-mcp.server
              -> bundle-mcp.connect
              -> bundle-mcp.tools-list

Security Impact (required)

New permissions/capabilities? (Yes/No) No
Secrets/tokens handling changed? (Yes/No) No
New/changed network calls? (Yes/No) No
Command/tool execution surface changed? (Yes/No) No
Data access scope changed? (Yes/No) No
If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

OS: Linux
Runtime/container: local checkout, Node.js via repo scripts
Model/provider: N/A for targeted MCP runtime verification
Integration/channel (if any): stdio MCP probe server
Relevant config (redacted): diagnostics timeline enabled via env; temporary MCP probe server path only

Steps

Enable diagnostics timeline output.
Start a real stdio MCP probe server through the bundle MCP runtime.
Materialize bundle MCP tools.
Read the timeline JSONL and assert per-server spans exist.

Expected

Timeline contains bundle-mcp.server, bundle-mcp.connect, and bundle-mcp.tools-list spans.

Actual

Timeline contains the expected spans and the targeted test passes.

Evidence

Failing test/log before + passing after
Trace/log snippets
Screenshot/recording
Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

Verified scenarios:
- Targeted agents test file passes: 12 passed.
- Focused new diagnostics test passes by itself.
- Formatting check passes for touched files.
- git diff --check passes.
Edge cases checked:
- Test restores diagnostics env vars after execution.
- Runtime is disposed after materialization.
What you did not verify:
- Full repo pnpm build && pnpm check && pnpm test, due memory pressure concerns on the available server.

Review Conversations

I replied to or resolved every bot review conversation I addressed in this PR.
I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

Backward compatible? (Yes/No) Yes
Config/env changes? (Yes/No) No
Migration needed? (Yes/No) No
If yes, exact upgrade steps: N/A

Risks and Mitigations

Risk: diagnostics output becomes slightly more verbose when timeline diagnostics are enabled.
- Mitigation: spans are only emitted through the existing diagnostics timeline gate and include only server names/transport type, not server URLs or headers.

clawsweeper · 2026-05-14T10:28:27Z

Codex review: needs changes before merge. Reviewed June 30, 2026, 9:22 PM ET / July 1, 2026, 01:22 UTC.

Summary
The PR adds diagnostics timeline spans around bundle MCP server startup, connection, and tools/list catalog discovery, plus focused runtime coverage.

PR surface: Source +37, Tests +63. Total +100 across 2 files.

Reproducibility: yes. at source level: current main still has only the aggregate bundle-tools prep marker while the active MCP connect and tools/list path lacks the requested per-server timeline spans. I did not run a live multi-server MCP repro during this read-only review.

Review metrics: 1 noteworthy metric.

Obsolete Touched Paths: 2 modified, 0 present on current main. Both PR-touched files are absent from current main, so maintainers need a port rather than ordinary review of the submitted path.

Stored data model
Persistent data-model change detected: serialized state: src/agents/pi-bundle-mcp-runtime.test.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #81595
Summary: This PR is the candidate implementation for the canonical bundle MCP diagnostics observability issue; adjacent performance reports would benefit from the signal but track broader or separate work.

Members:

canonical: observability: emit per-MCP-server sub-spans inside bundle-tools so cold-start cost is attributable #81595 - The PR body uses closing metadata and implements its requested per-server bundle MCP timeline spans.
partial_overlap: [2026.6.1 regression] bundle-tools prep stage 5–7s → 21–27s (3–4×) with 3 stdio MCP servers #91375 - That issue says these spans would help identify which MCP server regressed, but it tracks a separate bundle-tools latency regression.
partial_overlap: feat(embedded-runner): expose prep stage timings #78381 - That PR exposes higher-level prep stage timing to hook observers, while this PR adds lower-level bundle MCP timeline spans.
partial_overlap: [Performance]: Session switch blocks on synchronous MCP tool materialization (getCatalog + createRuntime) - 5-8s delay per switch #89408 - That issue covers session-switch delay from MCP materialization, overlapping the catalog path but asking for performance behavior beyond observability.
adjacent_distinct: [Analysis] Identify performance bottlenecks in native OpenClaw setup on constrained VPS #53600 - The broad native performance analysis is related observability context, not the same concrete runtime change.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦞 diamond lobster
Patch quality: 🧂 unranked krab
Result: blocked by patch quality or review findings.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Port the instrumentation and test to src/agents/agent-bundle-mcp-runtime.*.
Run node scripts/run-vitest.mjs src/agents/agent-bundle-mcp-runtime.test.ts and git diff --check after the port.

Risk before merge

[P1] A naive conflict resolution could reintroduce the old runtime shape or miss the active runtime entirely, leaving current bundle MCP discovery without the requested spans.
[P1] The current-main port must preserve session reuse, bounded fan-out, list/connect timeouts, unsupported-capability suppression, tool filtering, catalog invalidation, and existing failure diagnostics.
[P1] The contributor's proof demonstrates the intended spans on the submitted branch, but the port needs fresh focused validation on agent-bundle-mcp-runtime.* before merge.

Maintainer options:

Port Onto Active Runtime (recommended)
Move the span instrumentation and test to the current agent-bundle-mcp-runtime.* files while preserving active catalog/session semantics.
Wait For Author Refresh
Ask the contributor to rebase and port the same diagnostics idea manually if maintainers prefer author-owned follow-up over automated repair.

Copy recommended automerge instruction

@clawsweeper automerge

Special instructions:
Port the bundle MCP diagnostics timeline spans from obsolete `src/agents/pi-bundle-mcp-runtime.*` to current `src/agents/agent-bundle-mcp-runtime.ts` and `src/agents/agent-bundle-mcp-runtime.test.ts`; preserve session reuse, bounded fan-out, capability suppression, tool filtering, request timeout, catalog invalidation, failure diagnostics, and MCP protocol/config/tool materialization behavior; run `node scripts/run-vitest.mjs src/agents/agent-bundle-mcp-runtime.test.ts` and `git diff --check`.

Next step before merge

[P2] A focused automated repair can preserve the contributor's useful instrumentation by porting it from obsolete pi-bundle files to the active agent-bundle runtime and test.

Security
Cleared: The diff adds gated diagnostics instrumentation and a focused test only; no dependency, secret, permission, workflow, package-resolution, or new code-execution surface was found.

Review findings

[P1] Port the spans to the active bundle MCP runtime — src/agents/pi-bundle-mcp-runtime.ts:258-281

Review details

Best possible solution:

Port the gated timeline spans and focused stdio MCP coverage to src/agents/agent-bundle-mcp-runtime.* while preserving current MCP protocol, config, catalog, timeout, filtering, and tool materialization behavior.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main still has only the aggregate bundle-tools prep marker while the active MCP connect and tools/list path lacks the requested per-server timeline spans. I did not run a live multi-server MCP repro during this read-only review.

Is this the best way to solve the issue?

No, not as submitted. The instrumentation shape is narrow and maintainable, but it must be applied to the active agent-bundle-mcp runtime rather than obsolete pi-bundle files.

Full review comments:

[P1] Port the spans to the active bundle MCP runtime — src/agents/pi-bundle-mcp-runtime.ts:258-281
Current main no longer contains src/agents/pi-bundle-mcp-runtime.*; the active catalog path is src/agents/agent-bundle-mcp-runtime.ts. As submitted, this instruments a retired/conflicting path and would not add spans to current bundle MCP discovery.
Confidence: 0.95

Overall correctness: patch is incorrect
Overall confidence: 0.91

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against b3b51b0c9185.

Label changes

Label justifications:

P2: This is a bounded diagnostics improvement for attributing slow bundle MCP prep with limited runtime blast radius.
merge-risk: 🚨 compatibility: A naive merge or conflict resolution could bypass or replace the active bundle MCP runtime behavior with an obsolete implementation shape.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦞 diamond lobster and patch quality is 🧂 unranked krab.
status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body includes after-fix terminal proof from a local checkout running the real MCP runtime against a stdio MCP probe server and observing the expected timeline spans, though the current-main port needs fresh validation.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal proof from a local checkout running the real MCP runtime against a stdio MCP probe server and observing the expected timeline spans, though the current-main port needs fresh validation.

Evidence reviewed

PR surface:

Source +37, Tests +63. Total +100 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	62	25	+37
Tests	1	63	0	+63
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	125	25	+100

Acceptance criteria:

[P1] node scripts/run-vitest.mjs src/agents/agent-bundle-mcp-runtime.test.ts.
[P1] git diff --check.

What I checked:

Repository policy read: Root and scoped agent AGENTS.md were read fully; the review applied the runtime/current-main comparison and compatibility-sensitive agent hot-path guidance. (AGENTS.md:1, b3b51b0c9185)
Submitted patch target: The PR wraps the old pi-bundle connect and listTools path in measureDiagnosticsTimelineSpan and adds a timeline JSONL assertion test for the old module. (src/agents/pi-bundle-mcp-runtime.ts:258, f9ddbbe2d626)
Current active runtime: Current main has agent-bundle-mcp-runtime.* files and no pi-bundle-mcp-runtime.* files, so the branch does not instrument the active bundle MCP path as submitted. (src/agents/agent-bundle-mcp-runtime.ts:489, b3b51b0c9185)
Active connect/list path: The active runtime connects sessions through ensureSessionConnected and calls listAllToolsBestEffort with timeout, capability suppression, filtering, catalog invalidation, and bounded fan-out semantics that a port must preserve. (src/agents/agent-bundle-mcp-runtime.ts:699, b3b51b0c9185)
Aggregate marker still present: Current main still records a single aggregate bundle-tools prep stage before MCP tool materialization, so the linked observability gap remains unresolved on the active path. (src/agents/embedded-agent-runner/run/attempt.ts:1776, b3b51b0c9185)
No current span names: Focused source search found no bundle-mcp.server, bundle-mcp.connect, or bundle-mcp.tools-list span names on current main. (b3b51b0c9185)

Likely related people:

steipete: GitHub commit history shows substantial work on the older pi-bundle path and major refactors that shaped the current agent bundle MCP runtime boundary. (role: feature-history contributor; confidence: high; commits: 7043705ef3f3, 99ce71ddbbdb, bb46b79d3c14; files: src/agents/pi-bundle-mcp-runtime.ts, src/agents/agent-bundle-mcp-runtime.ts, src/agents/agent-bundle-mcp-runtime.test.ts)
vincentkoc: Recent history touches the active bundle MCP runtime/tests and introduced the shared diagnostics timeline helper that this PR should use on the current path. (role: recent area contributor; confidence: high; commits: 196ea61ec4d2, 80805ad7a583, dbc07ad84d2c; files: src/agents/agent-bundle-mcp-runtime.ts, src/agents/agent-bundle-mcp-runtime.test.ts, src/infra/diagnostics-timeline.ts)
mmyzwl: Authored the recent bundle-tools latency work that directly changed the active per-server connect/tools-list path this PR now needs to instrument. (role: recent performance contributor; confidence: medium; commits: a2725b6a24c8; files: src/agents/agent-bundle-mcp-runtime.ts, src/agents/agent-bundle-mcp-runtime.test.ts)
nxmxbbd: Merged history shows bounded tools/list timeout work in the old bundle MCP path, which is part of the behavior a current-path port must preserve. (role: adjacent MCP catalog contributor; confidence: medium; commits: 07f500aa562d; files: src/agents/pi-bundle-mcp-runtime.ts, src/agents/agent-bundle-mcp-runtime.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

openclaw-barnacle · 2026-06-02T05:09:08Z

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

Add bundle MCP timeline spans

f9ddbbe

openclaw-barnacle Bot added agents Agent runtime and tooling size: S proof: supplied External PR includes structured after-fix real behavior proof. labels May 14, 2026

clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 14, 2026

This comment was marked as low quality.

Sign in to view

clawsweeper Bot mentioned this pull request May 16, 2026

observability: emit per-MCP-server sub-spans inside bundle-tools so cold-start cost is attributable #81595

Open

openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jun 2, 2026

clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. labels Jun 2, 2026

dygg2001 mentioned this pull request Jun 8, 2026

[2026.6.1 regression] bundle-tools prep stage 5–7s → 21–27s (3–4×) with 3 stdio MCP servers #91375

Closed

openclaw-barnacle Bot removed the stale Marked as stale due to inactivity label Jun 15, 2026

This was referenced Jun 17, 2026

[Analysis] Identify performance bottlenecks in native OpenClaw setup on constrained VPS #53600

Open

feat(embedded-runner): expose prep stage timings #78381

Closed

clawsweeper Bot mentioned this pull request Jul 11, 2026

[Feature]: Intent-scoped MCP activation for large MCP catalogs #104312

Open

mushuiyu886 mentioned this pull request Jul 13, 2026

fix(mcp): clamp oversized MCP timeouts #105784

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add bundle MCP diagnostics timeline spans#81787

Add bundle MCP diagnostics timeline spans#81787
LLagoon3 wants to merge 1 commit into
openclaw:mainfrom
LLagoon3:fix-81595-bundle-tools-diagnostics

LLagoon3 commented May 14, 2026

Uh oh!

clawsweeper Bot commented May 14, 2026 •

edited

Loading

Uh oh!

This comment was marked as low quality.

openclaw-barnacle Bot commented Jun 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

LLagoon3 commented May 14, 2026

Summary

Change Type (select all)

Scope (select all touched areas)

Linked Issue/PR

Real behavior proof (required for external PRs)

Root Cause (if applicable)

Regression Test Plan (if applicable)

User-visible / Behavior Changes

Diagram (if applicable)

Security Impact (required)

Repro + Verification

Environment

Steps

Expected

Actual

Evidence

Human Verification (required)

Review Conversations

Compatibility / Migration

Risks and Mitigations

Uh oh!

clawsweeper Bot commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as low quality.

openclaw-barnacle Bot commented Jun 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented May 14, 2026 •

edited

Loading