Skip to content

docs(coding-agents): Claude Cowork application session tracing (TH-8296) - #933

Open
nik13 wants to merge 7 commits into
mainfrom
feat/th-8296-claude-cowork
Open

nik13 wants to merge 7 commits into
mainfrom
feat/th-8296-claude-cowork

Conversation

@nik13

@nik13 nik13 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Status 2026-10-21: ready for review. This supersedes the "BLOCKED-ON" wording below. Tracker result: DONE (replicated): real Claude Code 2.1.300 with Cowork's exporter settings, through the bridge into Future AGI (2026-10-20). Live Cowork needs a Team/Enterprise admin. The page says "Not yet run against a live Cowork organization", and an admin confirmation is optional, not a merge blocker. Merge after #912 (use a merge commit); this PR then retargets to main.


Summary

Claude Cowork session tracing (TH-8296) from Cowork's admin OpenTelemetry export. This is a BLOCKED-ON live verification draft (see the end).

Cowork exports log events, not traces. Per the monitoring reference: "Cowork exports events via the OTel logs/events protocol". The events are user_prompt, assistant_response, api_request, api_error, tool_result, and tool_decision, all linked by prompt.id. Future AGI ingests traces only, so examples/coding-agents/claude-cowork/bridge.py (stdlib only, OTLP/JSON in, OTLP/JSON out) turns each prompt.id into one trace, and the trace id is the prompt.id hex:

Cowork prompt (agent: session.id, prompt.id, account uuid as user.id, prompt totals under cowork.prompt.*)
├── Cowork model request (llm: model, tokens incl. cache, cost_usd -> gen_ai.cost.total; api_error -> ERROR "HTTP 429 rate_limit_error")
├── Read (tool: success, duration, approval decision/source, result size)
├── mcp__gmail__create_draft (tool_name "mcp_tool"; server and tool names read from tool_parameters)
├── Bash (tool_decision "reject" with no tool_result)
└── Cowork model request

Decision N-11 is implemented as an allow-list: spans are built only from named metadata fields, so nothing passes through. By default the bridge never forwards prompt, response, tool_input, tool error text, the API's error message, user.email, user.account_id, the device user.id, workspace.host_paths, enduser.id, or process.owner, even though a first-party Cowork export includes content by default. From tool_parameters it reads only mcp_server_name, mcp_tool_name, skill_name and subagent_type, and drops a value over 128 characters or containing /, \, @, a line break or a tab; commands, paths, URLs and every other key stay out. FI_BRIDGE_CAPTURE_CONTENT=1 adds the prompt, response, tool_input and error messages; identity fields stay out. Cowork pushes from Anthropic's cloud VM, so the bridge is internet-facing: it refuses to start without FI_BRIDGE_TOKEN (FI_BRIDGE_ALLOW_NO_TOKEN=1 for local tests), checks the token and path before reading a body, caps bodies at 16 MB, and the page keeps the loopback listen behind the reverse proxy.

Review round (2026-10-15), commit a6b52f1:

  • MCP/skill names from tool_parameters (names only); fixture uses tool_name: "mcp_tool" as Anthropic documents for user-configured MCP servers.
  • tool_result pairs with tool_decision by tool_use_id, else by name and decision value, so a rejected call survives a retried accepted one.
  • Prompt span ERROR only when its last model request failed (a retried 429/529 stays ERROR on its own span).
  • api_error status message is HTTP <code> <anthropic error type> by default; free text is content opt-in.
  • Identical re-delivered records are skipped (at-least-once delivery); prompt totals come from the deduped children.
  • session.id, gen_ai.conversation.id and user.id on every span.
  • Only service.name=cowork is mapped (PRD AC-04); FI_BRIDGE_ACCEPT_ANY_SERVICE=1 for engine tests, and the incoming service.name passes through.
  • HTTP: 411 without a length, 413 over 16 MB, chunked and gzip bodies, 30 s socket timeout, one log line per rejected status per minute.

Follow-up, commit bd38439:

  • The bearer token is compared as bytes: a non-ASCII Authorization header made hmac.compare_digest raise, so an unauthenticated client got a traceback and no reply instead of 401 (test added).
  • cowork.tool_calls counts only tools that ran: a tool_result with decision_type: reject is one reject span and is not counted (test added).
  • Page privacy table: with FI_BRIDGE_CAPTURE_CONTENT=1, tool input and tool error text carry commands, file paths, URLs and emails as Cowork sends them (the opt-in replay stores the fixture's /Users/fixture/... path in the Read span's input); the name filter's exact rule; the built-in-server name exception cites Claude Code's monitoring reference.

Final cleanup, commit 67dcdd8:

  • A tool_result pairs with a tool_decision only by tool_use_id, else by tool name plus decision_type. A result that carries no decision_type and matches no decision by tool_use_id now pairs with nothing. Before, it consumed the oldest same-name decision, so reject, accept, then such a result lost the reject span (recheck probe B7). Cowork's reference puts decision_type on every tool_result, so this only affects a non-conforming sender. Test added, with and without an unmatched tool_use_id; it fails on bd38439.
  • Page wording, checked against the Cowork monitoring reference:
    • The name filter also rejects a tab (now pinned by a test).
    • A rejected call's names come from tool_decision only when it carries tool_parameters. The reference lists only tool_name, decision and source there, so on real Cowork a rejected MCP call with no tool_result can show as mcp_tool.
    • user.id is on every span only when Cowork sends user.account_uuid. The reference says third-party deployments don't, so their spans have no user.id and don't appear in Users.
    • The description says content is stripped "by default".

Decision for Nikhil: PRD R-03 says the filter deletes tool_parameters. The bridge still drops the field, but reads four name keys out of it (MCP server/tool, skill, subagent type), because on real Cowork/Claude Code every MCP call otherwise shows as mcp_tool. PRD §3 keeps the tool name and §12 allows skill names that are not paths, so this follows the PRD's intent; say if you want names dropped too.

Files:

  • src/pages/docs/integrations/coding-agents/claude-cowork.mdx and its nav entry (after // coding-agents:claude-cowork).
  • examples/coding-agents/claude-cowork/: bridge.py, replay.py, run_e2e.sh, tests/test_bridge.py (23 unittest cases, Python 3.9 and 3.14), tests/fixtures/cowork-documented-session.jsonl (OTLP/JSON requests built field-for-field from the reference, content on as the first-party default sends it).

Checks at 67dcdd8: 23 unit tests pass on Python 3.9.6 and 3.14.4; npm run build passes; npm run audit-links: 0 broken nav or content links (29 orphan pages, unchanged). The recheck's mapper probe passes all 40 checks, and B7 now keeps the reject span.

How to run

FI_API_KEY=... FI_SECRET_KEY=... FI_PROJECT_NAME=cowork-sessions FI_BRIDGE_TOKEN=$(openssl rand -hex 32) \
  python3 examples/coding-agents/claude-cowork/bridge.py   # 127.0.0.1:4320, behind your HTTPS reverse proxy
# Admin settings > Cowork: endpoint https://<host>, protocol http/json, headers Authorization=Bearer <token>
python3 -m unittest discover -s examples/coding-agents/claude-cowork/tests
FI_API_KEY=... FI_SECRET_KEY=... ./examples/coding-agents/claude-cowork/run_e2e.sh   # documented-shape replay, no model call

E2E evidence (local Future AGI)

  1. Documented payloads through the real bridge, at bd38439. run_e2e.sh (token on, fresh ids) into a fresh project. fi_verify --project th8103-coding-claude-cowork-r3 --min-spans 9 --expect-type tool --expect-type llm --expect-type agent: PASS (2 traces / 9 spans):

    trace 201d53b2-06dd-4647-9ac3-f2f88f75277b (5 spans)
    - [agent] Cowork prompt model=claude-sonnet-5 (8000ms)
      - [llm] Cowork model request tok=10600/180 (3100ms)
      - [tool] Read (140ms)
      - [tool] Bash (0ms)                       <- rejected tool_decision
      - [llm] Cowork model request tok=10700/420 (3800ms)
    trace 4f7ce8d2-b99f-48be-b1f6-188cc6f16d94 (4 spans)
    - [agent] Cowork prompt (6000ms)            <- no ERROR: the 429 was retried successfully
      - [llm] Cowork model request status=ERROR:HTTP 429 rate_limit_error (900ms)
      - [llm] Cowork model request tok=10900/350 (3300ms)
      - [tool] mcp__gmail__create_draft status=ERROR:tool failed (210ms)
    PASS
    

    Every stored span (including LLM spans) has a session and user.id = the account UUID. The trace detail UI shows the prompt span with its two model requests, Read and the rejected Bash as children. Only these head-version projects are cited; the older th8103-coding-claude-cowork project holds traces from the pre-review bridge.

  2. Live event stream from the same engine (bridge at a6b52f1; bd38439 changes only the auth comparison and the rejected-result count). Real Claude Code 2.1.300 (claude -p, logs exporter, http/json, bearer token, OTEL_LOG_USER_PROMPTS=1 OTEL_LOG_TOOL_DETAILS=1 to mimic Cowork's content-on default) with a stub stdio MCP server gmail / create_draft, bridge with FI_BRIDGE_ACCEPT_ANY_SERVICE=1. Claude Code sent the MCP call as tool_name: "mcp_tool" with tool_parameters {"mcp_server_name":"gmail","mcp_tool_name":"create_draft"}, and tool_use_id on decisions and results. Project th8103-coding-claude-cowork-r2-engine: PASS (trace 6f38fc41-9707-4fd8-83ca-7e11fd2efd77, 7 spans: prompt → llm 31107/175 tok, Bash, Read, llm, mcp__gmail__create_draft, llm). This is not Cowork evidence; it shows the bridge handles the real envelope, real MCP naming and real timing.

    67dcdd8 was not re-run against local Future AGI. It changes only the pairing of a tool_result with no decision_type and the page wording. Every tool event in the fixture carries a decision, and the unit tests that pin the fixture's span trees pass unchanged.

  3. AC-04 negative. The same Claude Code setup against a bridge without the test flag: the bridge logged ignoring events from service.name=claude-code and exported nothing.

Privacy proof

  • Default (fixture, th8103-coding-claude-cowork-r3, bd38439): PROMPT_MARKER_COWORK, RESPONSE_MARKER_COWORK, TOOL_INPUT_MARKER_COWORK, ERROR_MARKER_COWORK, the API error text, [email protected], [email protected], /Users/fixture, fixture-login, user_01BWBeN28fixture, tool_parameters, file_path, and this Mac's username, home path and hostname: 0 matches in the stored spans.
  • Default (live stream): the raw OTLP input had the prompt marker ×2, the draft subject ×4, the ls command ×5 and a local /Users/... path ×1; the stored spans have 0 of each, 0 bash_command, 0 tool_parameters, and the MCP names only as cowork.mcp_server_name=gmail / cowork.mcp_tool_name=create_draft.
  • Opt-in (th8103-coding-claude-cowork-r3-content, bd38439, fi_verify PASS): the prompt, response, tool-input and error markers and the API error text are present (the Read span's input holds the fixture's file path, as the page now says); email, workspace.host_paths, login, user.account_id and tool_parameters are still absent.
  • Unit tests cover an api_error whose text holds a path, a username and an internal host (absent by default, present with opt-in) and tool_parameters with commands and paths (absent in both modes).

Limitations / not available

  • No span hierarchy below the prompt (the events don't link tools to model requests). Skills and plugins have no documented event of their own; a skill call is a tool span with cowork.skill_name. The otlpTracesEnabled beta (third-party deployments) is not covered.
  • MCP/skill names come only from tool_parameters; with otlpContentCapture=[] Cowork may leave it out, and MCP calls then show as mcp_tool (skill calls as Skill). The reference lists no tool_parameters on tool_decision, so a rejected MCP call with no tool_result can show as mcp_tool.
  • On third-party deployments Cowork sends no user.account_uuid, so spans have no user.id (enduser.id is dropped as identity).
  • The record envelope (event.name attribute vs. a prefixed body) isn't specified in the reference. The bridge accepts user_prompt, cowork.user_prompt, and claude_code.user_prompt (tested).
  • Delivery is at least once; identical re-sent records are skipped.
  • Model response text and cost come only from what Cowork sends. Cost is Anthropic's estimate.

How other tools support Cowork (read 2026-10-20)

Tool Cowork? Mechanism Plan Source
Anthropic Yes Native OTLP logs/events from Admin settings > Cowork; beta trace export on third-party deployments Team/Enterprise, admin https://claude.com/docs/cowork/monitoring
Elastic Yes (write-up) OTLP to an OTel gateway Team/Enterprise https://www.elastic.co/security-labs/blog/claude-code-cowork-monitoring-otel-elastic
Honeycomb, Datadog Named by Anthropic as backends; no Cowork page Generic OTLP logs Team/Enterprise same Anthropic page; https://docs.honeycomb.io/send-data/use-cases/claude-platform ; https://docs.datadoghq.com/ai_agents_console/setup/
SigNoz, Grafana, New Relic, Dynatrace, PostHog No Cowork page found Claude Code only (SigNoz https://signoz.io/docs/claude-code-monitoring ; New Relic Claude Platform traces https://newrelic.com/fr/blog/news/anthropic-otel-ai-observability) n/a
Langfuse, LangSmith, Braintrust, Arize, Laminar, Opik, Helicone No Cowork integration found Claude Code via hooks/gateway (https://langfuse.com/resources/engineering/coding-agent-tracing) n/a

No vendor demos Cowork without an admin. Cowork's events use Claude Code's event names and prompt.id correlation (Cowork adds assistant_response, workspace.host_paths, service.name=cowork), so Claude Code is the shared stand-in. Page updated with this table and a Verification section (0e68a29).

Verification (2026-10-20), bridge at 0e68a29 (code unchanged since 67dcdd8)

  • Real: Claude Code 2.1.300 with the exporter settings the admin form sets (OTLP logs, http/json, Authorization=Bearer, prompt/tool details on) -> tee -> bridge (FI_BRIDGE_ACCEPT_ANY_SERVICE=1) -> local Future AGI. Prompt: ls, Read README.md, MCP gmail.create_draft. fi_verify --project th8103-livefill-cowork: PASS, trace b2f51f94-bd9a-4d8b-9a23-5b0930880bf2, 8 spans (prompt; model requests 16617/259, 17117/249, 17378/33 tokens; Bash; ToolSearch; Read; mcp__gmail__create_draft).
  • Privacy scan of every stored spans column with unique markers: prompt marker, draft subject, draft body, /Users/, login name, host name, tool_parameters, @: 0 matches each. The raw OTLP stream had the prompt marker x2 and the draft subject x3.
  • Stand-in: service.name is claude-code here (test flag); Claude Code sends no assistant_response or workspace.host_paths. Documented-shape replay (run_e2e.sh) results above are unchanged.
  • Docs build and link audit: exit 0, 0 broken links.

BLOCKED-ON (reduced)

Only the real-Cowork parts remain: a Team/Enterprise admin enabling Admin settings > Cowork with the bridge URL and Authorization=Bearer <FI_BRIDGE_TOKEN>, to confirm Cowork's exact record envelope and post path, real tool_parameters for connectors and skills, and delivery from Anthropic's VM. Everything else is covered above. No paid plan or admin access was used.

🤖 Generated with Claude Code

nik13 and others added 4 commits October 14, 2026 21:42
…tent stripped (TH-8296)

Cowork's admin OpenTelemetry export (Team/Enterprise) sends log events
(user_prompt, assistant_response, api_request, api_error, tool_result,
tool_decision) linked by prompt.id; Future AGI ingests traces only. A
stdlib bridge turns each prompt.id into one trace (trace id = prompt.id):
prompt (agent) -> model requests (llm: tokens, cache, cost_usd, errors)
and tool calls (tool: success, duration, approval; rejected decisions).

Decision N-11: spans are built from an allow-list of metadata fields.
Prompt, response, tool_input, tool_parameters, tool error text,
user.email, workspace.host_paths, enduser.id and process.owner are never
forwarded by default; FI_BRIDGE_CAPTURE_CONTENT=1 adds prompt, response
and tool_input only. Internet-facing bridge requires a bearer token.

- page, nav entry
- examples/coding-agents/claude-cowork: bridge.py, replay.py, run_e2e.sh,
  unittest contract tests on payloads built from Anthropic's Cowork
  monitoring reference

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…tries, dedupe, HTTP front (TH-8296)

- Name MCP tool spans mcp__<server>__<tool> from tool_parameters; read only
  mcp_server_name, mcp_tool_name, skill_name and subagent_type from it, and
  drop a value that looks like a path or email. Fixture now uses tool_name
  "mcp_tool" as Anthropic documents for user-configured MCP servers.
- Pair tool_result with tool_decision by tool_use_id, else by name and
  decision value, so a rejected call survives a retried accepted one.
- Mark the prompt ERROR only when its last model request failed; a retried
  429/529 stays ERROR on its own span.
- api_error status message is "HTTP <code> <anthropic error type>" by
  default; the API's free text needs FI_BRIDGE_CAPTURE_CONTENT=1.
- Skip identical re-delivered records; compute prompt totals and tool count
  from the deduped children.
- Copy session.id, gen_ai.conversation.id and user.id to every span.
- Map only service.name=cowork (FI_BRIDGE_ACCEPT_ANY_SERVICE=1 for tests);
  pass the incoming service.name through.
- HTTP: check token and path before reading the body, 411 without a length,
  413 over 16 MB, chunked and gzip bodies, 30 s socket timeout, log rejected
  requests once per status per minute.
- Require FI_BRIDGE_TOKEN always (FI_BRIDGE_ALLOW_NO_TOKEN=1 for local tests);
  page keeps the loopback listen behind the reverse proxy.
- Let an in-flight send finish on shutdown; tolerate non-numeric
  event.sequence; run_e2e.sh accepts a fractional idle time and cleans up.
- Page: what is captured, privacy table, not-available and troubleshooting
  (404/411/413/415, ignored service) match the code.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…rivacy wording (TH-8296)

- Compare the bearer token as bytes: hmac.compare_digest raised TypeError on a
  non-ASCII Authorization header, so an unauthenticated client got a
  traceback and no reply instead of 401.
- cowork.tool_calls counts only tools that ran: a tool_result whose decision
  was reject is one reject span and is not counted.
- Page: with FI_BRIDGE_CAPTURE_CONTENT=1, tool input and tool error text carry
  commands, file paths, URLs and emails as Cowork sends them; the
  tool_parameters name filter's exact rule (128 chars, no / \ @ or line
  break); the built-in-server name exception cites Claude Code's reference.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…; page nits (TH-8296)

- Pair a tool_result with its tool_decision only by tool_use_id, else by tool
  name plus decision_type. A result that carries no decision_type and matches
  no decision by tool_use_id now pairs with nothing, so an earlier reject keeps
  its own span instead of being consumed as the oldest same-name decision.
  Test covers the result with and without an unmatched tool_use_id.
- Page: the pairing rule says what happens to such a result; the name filter
  also rejects tab (pinned by a test); a rejected MCP call's names appear only
  when tool_decision carries tool_parameters, which Cowork's reference does
  not list; user.id is on spans only when Cowork sends the account UUID
  (absent on third-party deployments); the description says "by default".

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@nik13
nik13 marked this pull request as ready for review October 21, 2026 22:59
Base automatically changed from feat/th-8229-cursor-hook-to-otlp to main October 29, 2026 05:37
nik13 and others added 2 commits October 29, 2026 03:42
Resolve src/lib/navigation.ts by keeping main's catalog-driven
Integrations sidebar; the page is listed through src/lib/integrations.mjs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant