Skip to content

Commit 5db2f6c

Browse files
jesse-merhijesse-merhi
authored
Add stdout diagnostics OTEL log exporter
Adds stdout and both-mode diagnostics OTEL log export, with focused QA Lab smoke coverage and docs/config updates. Prepared head SHA: efa2ef0 Verification: CI 27808480969 passed for the prepared head. Reviewed-by: @jesse-merhi
1 parent afd9cb0 commit 5db2f6c

27 files changed

Lines changed: 1133 additions & 155 deletions

‎docs/.generated/config-baseline.sha256‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
e78623d6eace69e46950cd5d9a5cf14aa910dac1ecdf9d054a0bd9999e936061 config-baseline.json
2-
5ecafa3c9a59fc0675f964f6e3238b2f20625376ebad1835278c5dd7323770d3 config-baseline.core.json
1+
ac06b6c20a93a8543ec1bd3748ef4f7bdae5006839dd93b3fff874d0da4244aa config-baseline.json
2+
e7965566fdaedef445bcd562141f4f3ea1a499cf8ea5956418af7c98049bf242 config-baseline.core.json
33
2d735389858305509528e74329b6f8c65d311e1471c3b4e91dc17aaab8e63a80 config-baseline.channel.json
4-
7c2c51b795d32e4c4c325080d59fec8fd11317c41db7db642f70e436779738bc config-baseline.plugin.json
4+
0039da0cf2ba2845b37db52c4cf3a0f25e367cf3d2d507c5d6f8a5e5bdfdc4d4 config-baseline.plugin.json

‎docs/gateway/configuration-reference.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1096,6 +1096,7 @@ Notes:
10961096
traces: true,
10971097
metrics: true,
10981098
logs: false,
1099+
logsExporter: "otlp",
10991100
sampleRate: 1.0,
11001101
flushIntervalMs: 5000,
11011102
captureContent: {
@@ -1132,6 +1133,7 @@ Notes:
11321133
- `otel.headers`: extra HTTP/gRPC metadata headers sent with OTel export requests.
11331134
- `otel.serviceName`: service name for resource attributes.
11341135
- `otel.traces` / `otel.metrics` / `otel.logs`: enable trace, metrics, or log export.
1136+
- `otel.logsExporter`: log export sink: `"otlp"` (default), `"stdout"` for one JSON object per stdout line, or `"both"`.
11351137
- `otel.sampleRate`: trace sampling rate `0`-`1`.
11361138
- `otel.flushIntervalMs`: periodic telemetry flush interval in ms.
11371139
- `otel.captureContent`: opt-in raw content capture for OTEL span attributes. Defaults to off. Boolean `true` captures non-system message/tool content; the object form lets you enable `inputMessages`, `outputMessages`, `toolInputs`, `toolOutputs`, `systemPrompt`, and `toolDefinitions` explicitly.

‎docs/gateway/opentelemetry.md‎

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
summary: "Export OpenClaw diagnostics to any OpenTelemetry collector via the diagnostics-otel plugin (OTLP/HTTP)"
2+
summary: "Export OpenClaw diagnostics to OpenTelemetry collectors or stdout JSONL via the diagnostics-otel plugin"
33
title: "OpenTelemetry export"
44
read_when:
55
- You want to send OpenClaw model usage, message flow, or session metrics to an OpenTelemetry collector
@@ -8,17 +8,19 @@ read_when:
88
---
99

1010
OpenClaw exports diagnostics through the official `diagnostics-otel` plugin
11-
using **OTLP/HTTP (protobuf)**. Any collector or backend that accepts OTLP/HTTP
12-
works without code changes. For local file logs and how to read them, see
13-
[Logging](/logging).
11+
using **OTLP/HTTP (protobuf)**. Logs can also be written as stdout JSONL for
12+
container and sandbox log pipelines. Any collector or backend that accepts
13+
OTLP/HTTP works without code changes. For local file logs and how to read them,
14+
see [Logging](/logging).
1415

1516
## How it fits together
1617

1718
- **Diagnostics events** are structured, in-process records emitted by the
1819
Gateway and bundled plugins for model runs, message flow, sessions, queues,
1920
and exec.
2021
- **`diagnostics-otel` plugin** subscribes to those events and exports them as
21-
OpenTelemetry **metrics**, **traces**, and **logs** over OTLP/HTTP.
22+
OpenTelemetry **metrics**, **traces**, and **logs** over OTLP/HTTP. It can
23+
also mirror diagnostic log records to stdout JSONL.
2224
- **Provider calls** receive a W3C `traceparent` header from OpenClaw's
2325
trusted model-call span context when the provider transport accepts custom
2426
headers. Plugin-emitted trace context is not propagated.
@@ -74,11 +76,13 @@ openclaw plugins enable diagnostics-otel
7476
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
7577
| **Metrics** | Counters and histograms for token usage, cost, run duration, failover, skill usage, message flow, Talk events, queue lanes, session state/recovery, tool execution, oversized payloads, exec, and memory pressure. |
7678
| **Traces** | Spans for model usage, model calls, harness lifecycle, skill usage, tool execution, exec, webhook/message processing, context assembly, and tool loops. |
77-
| **Logs** | Structured `logging.file` records exported over OTLP when `diagnostics.otel.logs` is enabled; log bodies are withheld unless content capture is explicitly enabled. |
79+
| **Logs** | Structured `logging.file` records exported over OTLP or stdout JSONL when `diagnostics.otel.logs` is enabled; log bodies are withheld unless content capture is explicitly enabled. |
7880

7981
Toggle `traces`, `metrics`, and `logs` independently. Traces and metrics
8082
default to on when `diagnostics.otel.enabled` is true. Logs default to off and
81-
are exported only when `diagnostics.otel.logs` is explicitly `true`.
83+
are exported only when `diagnostics.otel.logs` is explicitly `true`. Log export
84+
defaults to OTLP; set `diagnostics.otel.logsExporter` to `stdout` for JSONL on
85+
stdout, or `both` to send each diagnostic log record to OTLP and stdout.
8286

8387
## Configuration reference
8488

@@ -98,6 +102,7 @@ are exported only when `diagnostics.otel.logs` is explicitly `true`.
98102
traces: true,
99103
metrics: true,
100104
logs: true,
105+
logsExporter: "otlp", // otlp | stdout | both
101106
sampleRate: 0.2, // root-span sampler, 0.0..1.0
102107
flushIntervalMs: 60000, // metric export interval (min 1000ms)
103108
captureContent: {
@@ -176,6 +181,11 @@ on the public diagnostic event bus.
176181
- **Logs:** OTLP logs respect `logging.level` (file log level). They use the
177182
diagnostic log-record redaction path, not console formatting. High-volume
178183
installs should prefer OTLP collector sampling/filtering over local sampling.
184+
Set `diagnostics.otel.logsExporter: "stdout"` when your platform already
185+
ships stdout/stderr to a log processor and you do not have an OTLP logs
186+
collector. Stdout records are one JSON object per line with `ts`, `signal`,
187+
`service.name`, severity, body, redacted attributes, and trusted trace fields
188+
when available.
179189
- **File-log correlation:** JSONL file logs include top-level `traceId`,
180190
`spanId`, `parentSpanId`, and `traceFlags` when the log call carries a valid
181191
diagnostic trace context, which lets log processors join local log lines with

‎docs/logging.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -224,8 +224,10 @@ model-call traces become children of the active request trace, so local logs,
224224
diagnostic snapshots, OTEL spans, and trusted provider `traceparent` headers can
225225
be joined by `traceId` without logging raw request or model content.
226226

227-
Talk lifecycle log records also flow to OTLP logs when OpenTelemetry log export
228-
is enabled, using the same bounded attributes as file logs.
227+
Talk lifecycle log records also flow to diagnostics-otel log export when
228+
OpenTelemetry log export is enabled, using the same bounded attributes as file
229+
logs. Configure `diagnostics.otel.logsExporter` to choose OTLP, stdout JSONL, or
230+
both sinks.
229231

230232
### Model call size and timing
231233

‎docs/plugins/plugin-inventory.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -227,7 +227,7 @@ Each entry lists the package, distribution route, and description.
227227

228228
- **[deepseek](/plugins/reference/deepseek)** (`@openclaw/deepseek-provider`) - npm; ClawHub: `clawhub:@openclaw/deepseek-provider`. Adds DeepSeek model provider support to OpenClaw.
229229

230-
- **[diagnostics-otel](/plugins/reference/diagnostics-otel)** (`@openclaw/diagnostics-otel`) - npm; ClawHub: `clawhub:@openclaw/diagnostics-otel`. OpenClaw diagnostics OpenTelemetry exporter for metrics and traces.
230+
- **[diagnostics-otel](/plugins/reference/diagnostics-otel)** (`@openclaw/diagnostics-otel`) - npm; ClawHub: `clawhub:@openclaw/diagnostics-otel`. OpenClaw diagnostics OpenTelemetry exporter for metrics, traces, and logs.
231231

232232
- **[diagnostics-prometheus](/plugins/reference/diagnostics-prometheus)** (`@openclaw/diagnostics-prometheus`) - npm; ClawHub: `clawhub:@openclaw/diagnostics-prometheus`. OpenClaw diagnostics Prometheus exporter for runtime metrics.
233233

‎docs/plugins/reference/diagnostics-otel.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
---
2-
summary: "OpenClaw diagnostics OpenTelemetry exporter for metrics and traces."
2+
summary: "OpenClaw diagnostics OpenTelemetry exporter for metrics, traces, and logs."
33
read_when:
44
- You are installing, configuring, or auditing the diagnostics-otel plugin
55
title: "Diagnostics OpenTelemetry plugin"
66
---
77

88
# Diagnostics OpenTelemetry plugin
99

10-
OpenClaw diagnostics OpenTelemetry exporter for metrics and traces.
10+
OpenClaw diagnostics OpenTelemetry exporter for metrics, traces, and logs.
1111

1212
## Distribution
1313

‎extensions/diagnostics-otel/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Official OpenTelemetry diagnostics exporter for OpenClaw.
44

5-
This plugin exports OpenClaw Gateway traces, metrics, and logs to an OTLP collector for observability stacks such as Grafana, Datadog, Honeycomb, New Relic, Tempo, and compatible collectors.
5+
This plugin exports OpenClaw Gateway traces, metrics, and logs to an OTLP collector for observability stacks such as Grafana, Datadog, Honeycomb, New Relic, Tempo, and compatible collectors. It can also write diagnostic log records as stdout JSONL for container log pipelines.
66

77
## Install
88

‎extensions/diagnostics-otel/openclaw.plugin.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"id": "diagnostics-otel",
33
"name": "Diagnostics OpenTelemetry",
4-
"description": "OpenClaw diagnostics OpenTelemetry exporter for metrics and traces.",
4+
"description": "OpenClaw diagnostics OpenTelemetry exporter for metrics, traces, and logs.",
55
"activation": {
66
"onStartup": true
77
},

‎extensions/diagnostics-otel/package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "@openclaw/diagnostics-otel",
33
"version": "2026.6.8",
4-
"description": "OpenClaw diagnostics OpenTelemetry exporter for metrics and traces.",
4+
"description": "OpenClaw diagnostics OpenTelemetry exporter for metrics, traces, and logs.",
55
"repository": {
66
"type": "git",
77
"url": "https://github.com/openclaw/openclaw"

0 commit comments

Comments
 (0)