Skip to content

feat(telemetry): add OpenTelemetry instrumentation with Aspire Dashboard support#6629

Draft
Hona wants to merge 227 commits intoanomalyco:devfrom
Hona:feat/aspire-otel
Draft

feat(telemetry): add OpenTelemetry instrumentation with Aspire Dashboard support#6629
Hona wants to merge 227 commits intoanomalyco:devfrom
Hona:feat/aspire-otel

Conversation

@Hona
Copy link
Copy Markdown
Member

@Hona Hona commented Jan 2, 2026

Adds experimental OpenTelemetry support for debugging and observability.

What

  • Full OTEL instrumentation: all tools, MCP, sessions, LLM, LSP, plugins
  • Aspire Dashboard integration via bun run dev:otel
  • Service differentiation: opencode-cli vs opencode-server
  • Structured logs with full key=value context + exception stack traces
  • AI SDK telemetry with GenAI message content capture

Enabling OpenTelemetry

  1. Add to your global config (~/.config/opencode/opencode.json:
{
  "experimental": {
    "openTelemetry": true
  }
}
  1. Run with Aspire Dashboard:
cd packages/opencode
bun run dev:otel
  1. Open dashboard at http://localhost:18888

The OTEL_EXPORTER_OTLP_ENDPOINT env var controls the endpoint (defaults to http://localhost:4317).

Images

image image image image image image image

Hona added 30 commits January 2, 2026 14:11
Change experimental.openTelemetry config from boolean to union type
supporting both boolean and object with enabled/endpoint fields.
This allows users to configure custom OTLP endpoints for Aspire Dashboard
integration while maintaining backward compatibility with boolean config.
…tion

Add telemetry module with:
- Config interface and resolveConfig() for endpoint resolution
- init() function with NodeSDK, LoggerProvider, trace/log exporters
- shutdown() for graceful cleanup
- withSpan() helper for span creation with error handling
- isEnabled(), getTracer(), getLogger() utility functions
- SeverityMap for log level mapping
Integrate OpenTelemetry log emission into the Log module. When telemetry
is enabled, all log messages (debug/info/warn/error) are emitted to the
OTLP endpoint alongside file-based logging.

- Lazy-load telemetry module to avoid circular dependency
- Guard against recursive calls during module initialization
- Emit logs with proper severity levels using Telemetry.SeverityMap
- Initialize telemetry in yargs middleware after Log.init()
- Check OTEL_EXPORTER_OTLP_ENDPOINT env var or config.experimental.openTelemetry
- Register SIGTERM and SIGINT handlers for graceful shutdown
- Call Telemetry.shutdown() in finally block before process.exit()
Hona added 21 commits March 23, 2026 21:59
…re Dashboard GenAI support

Add 150+ spans across the entire distributed architecture:

GenAI Features:
- gen_ai.system, gen_ai.operation.name, gen_ai.request.model attributes
- gen_ai.usage.input_tokens/output_tokens for token tracking
- gen_ai.tool.definitions for tool visualization
- gen_ai.response.id and gen_ai.response.model
- GenAI events: gen_ai.system.message, gen_ai.user.message, gen_ai.assistant.message

Distributed Architecture:
- Worker lifecycle spans (init, startup, shutdown)
- Resource naming: opencode-cli, opencode-worker, opencode-lsp-server
- Span linking across thread boundaries with trace context propagation
- Messaging spans: messaging.send, messaging.receive, messaging.process
- IPC/Named pipe spans for inter-process communication

Database Layer:
- SQLite operation spans: db.session.insert, db.session.select, db.session.update
- Migration spans with statistics
- db.operation wrapper for all queries

HTTP & Network:
- http.request spans for API routes
- http.download spans for LSP server downloads (GitHub releases)
- archive.extract spans for zip/tar extraction
- External dependency visibility

Security:
- OAuth flow spans: oauth.flow.start, oauth.callback.receive, oauth.token.store
- MCP authentication spans
- Token validation and refresh tracking

Background Operations:
- scheduler.task.execute spans
- queue.work spans with depth tracking
- file.watcher.event spans
- server.heartbeat spans

LSP Integration:
- lsp.request.* spans for all JSON-RPC operations
- lsp.server.spawn spans with process details
- lsp.notification spans
- code.intelligence spans

Tool Execution:
- tool.websearch.execute with search parameters
- tool.webfetch.execute with retry tracking
- tool.read.file with binary detection
- tool.write.execute with diff statistics
- tool.edit.execute with change metrics
- tool.list.execute with file counts
- tool.apply_patch.execute with hunk statistics
- tool.bash.spawn, tool.grep.spawn, tool.git.execute

Event System:
- event.publish spans
- event.handle spans with subscriber counts
- bus.subscribe spans
- ipc.emit/receive spans

TUI Operations:
- tui.render spans
- tui.input spans
- command.execute spans
- action.execute spans

All spans follow OpenTelemetry semantic conventions and enable full
Aspire Dashboard 13.2 GenAI visualization including:
- Resource list showing distributed components
- GenAI visualizer with message timeline
- Tools tab with definitions and parameters
- Token usage tracking
- Complete trace waterfall across threads
# Conflicts:
#	.opencode/opencode.jsonc
#	bun.lock
#	packages/opencode/package.json
#	packages/opencode/src/agent/agent.ts
#	packages/opencode/src/bus/index.ts
#	packages/opencode/src/cli/cmd/tui/app.tsx
#	packages/opencode/src/cli/cmd/tui/thread.ts
#	packages/opencode/src/cli/cmd/tui/worker.ts
#	packages/opencode/src/file/watcher.ts
#	packages/opencode/src/flag/flag.ts
#	packages/opencode/src/index.ts
#	packages/opencode/src/lsp/client.ts
#	packages/opencode/src/lsp/index.ts
#	packages/opencode/src/lsp/server.ts
#	packages/opencode/src/mcp/auth.ts
#	packages/opencode/src/plugin/index.ts
#	packages/opencode/src/scheduler/index.ts
#	packages/opencode/src/server/server.ts
#	packages/opencode/src/session/compaction.ts
#	packages/opencode/src/session/index.ts
#	packages/opencode/src/session/llm.ts
#	packages/opencode/src/session/processor.ts
#	packages/opencode/src/session/prompt.ts
#	packages/opencode/src/session/summary.ts
#	packages/opencode/src/shell/shell.ts
#	packages/opencode/src/snapshot/index.ts
#	packages/opencode/src/storage/db.ts
#	packages/opencode/src/storage/json-migration.ts
#	packages/opencode/src/tool/bash.ts
#	packages/opencode/src/tool/edit.ts
#	packages/opencode/src/tool/grep.ts
#	packages/opencode/src/tool/read.ts
#	packages/opencode/src/tool/tool.ts
#	packages/opencode/src/tool/webfetch.ts
#	packages/opencode/src/tool/write.ts
#	packages/opencode/src/util/git.ts
…m/dev

Re-apply all OpenTelemetry instrumentation on top of upstream dev (1205 commits merged).

Key instrumentation:
- GenAI attributes on LLM spans (gen_ai.system, gen_ai.operation.name, etc.)
- Token usage tracking (gen_ai.usage.input_tokens/output_tokens)
- Database spans with db.system: sqlite for Aspire Database filter
- HTTP spans with METHOD /route naming
- Tool execution wrapper spans (tool.execute)
- Individual tool spans (read, write, edit, bash, grep, etc.)
- MCP, LSP, OAuth, plugin, snapshot spans
- Context propagation fix for proper span nesting
- Aspire Dashboard launch scripts and MCP config
…at span via onFinish, include tool parameters
@bbartels
Copy link
Copy Markdown
Contributor

bbartels commented Mar 24, 2026

@Hona Saw your presentation at aspire conf yesterday! Was quite keen to try out the Aspire Dashboard for aspire. Would this PR be needed to get this to work?

And would this work with the final built artefact as opposed to just with the dev environment for opencode? We would mainly be interested in it to help us track down issues with our installations of opencode

@slang25
Copy link
Copy Markdown

slang25 commented Mar 24, 2026

I saw the talk too and really want this 😅 are there plans for this to be merged?
Great talk by the way

@maxious
Copy link
Copy Markdown
Contributor

maxious commented Mar 25, 2026

FYI robo-dax just closed #2666 and #6142 which I was also interested in for otel monitoring of opencode. Will this PR include those features?

@github-actions github-actions bot mentioned this pull request Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants