Skip to content

Commit 29c94a0

Browse files
Aryansharma28claude
andcommitted
feat(langy): trace opencode worker activity into the user's project (tagged "langy")
opencode has no native OpenTelemetry export (anomalyco/opencode#12142, #14246, #14697), so the per-session worker now loads the pinned @devtheops/opencode-plugin-otel plugin, which emits session/llm/tool spans over OTLP http/protobuf to LangWatch's existing /api/otel/v1/traces, authenticated with the project's dedicated Langy key (resolves its own project — no X-Project-Id). The plugin tags via OTLP *resource* attributes, but ingestion only mapped the reserved tag.tags->labels / langwatch.thread.id->thread_id keys from *span* attributes. So otel.traces.ts now hoists reserved metadata keys found in resource attributes too (coercing tag.tags to a string[] for labels), additively and guarded so a malformed resource attribute can never break ingestion. This is what makes Langy's traces land labeled "langy" and grouped by conversation. No new REST endpoints (reuses /api/otel/v1/traces). Spec + resource-attribute integration tests added. Closes #4536. Part of epic #4528. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
1 parent bc0efea commit 29c94a0

5 files changed

Lines changed: 266 additions & 4 deletions

File tree

‎Dockerfile.langy_agent‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,11 @@ FROM ubuntu:24.04
1414
# in lockstep with a tested release rather than tracking `latest`.
1515
ARG LANGWATCH_CLI_VERSION=0.31.0
1616
ARG LANGWATCH_MCP_VERSION=0.10.1
17+
# opencode has no native OpenTelemetry export, so the per-session worker loads
18+
# this opencode plugin to emit session/llm/tool spans over OTLP into the user's
19+
# LangWatch project. Pinned; opencode auto-loads it by name from the per-worker
20+
# config (server.js). See specs/assistant/langy-otel-tracing.feature.
21+
ARG OPENCODE_OTEL_PLUGIN_VERSION=1.0.0
1722

1823
RUN apt-get update && apt-get install -y --no-install-recommends \
1924
curl git ca-certificates unzip build-essential python3 \
@@ -32,9 +37,16 @@ RUN curl -fsSL https://opencode.ai/install | bash && \
3237

3338
# CLI (`langwatch`) + MCP server (`langwatch-mcp-server`) from npm. Replaces
3439
# the old locally-`pnpm pack`'d tarballs so the image builds reproducibly in
35-
# CI with no host-side build step.
40+
# CI with no host-side build step. The opencode OTel plugin is installed here
41+
# too so the pinned version is present in the image (and the build fails fast
42+
# if the version is bad) rather than relying solely on opencode's runtime fetch.
3643
RUN npm install -g "langwatch@${LANGWATCH_CLI_VERSION}" \
37-
"@langwatch/mcp-server@${LANGWATCH_MCP_VERSION}"
44+
"@langwatch/mcp-server@${LANGWATCH_MCP_VERSION}" \
45+
"@devtheops/opencode-plugin-otel@${OPENCODE_OTEL_PLUGIN_VERSION}"
46+
47+
# Surface the pinned plugin version to the manager (server.js) so each
48+
# per-worker opencode config requests the exact same version opencode loads.
49+
ENV OPENCODE_OTEL_PLUGIN_VERSION=${OPENCODE_OTEL_PLUGIN_VERSION}
3850

3951
COPY services/langy-agent/entrypoint.sh /entrypoint.sh
4052
COPY services/langy-agent/server.js /server.js

‎langwatch/src/server/tracer/__tests__/metadataLabels.integration.test.ts‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -859,3 +859,101 @@ describe("Integration - Real-world SDK scenarios", () => {
859859
expect(trace.spans[0]?.type).toBe("llm");
860860
});
861861
});
862+
863+
/**
864+
* Helper to create a minimal OTEL trace request with the given OTLP *resource*
865+
* attributes (as opposed to span attributes). This mirrors how an OTel SDK
866+
* serialises OTEL_RESOURCE_ATTRIBUTES — flat string-valued key/value pairs on
867+
* the resource shared by every span. Used to verify that reserved metadata
868+
* keys are mapped from resource attributes, not only span attributes.
869+
*/
870+
function createOtelTraceWithResourceAttributes(
871+
resourceAttributes: Array<{ key: string; value: { stringValue?: string } }>,
872+
traceId = "dGVzdC10cmFjZS1pZDEyMzQ=",
873+
): DeepPartial<IExportTraceServiceRequest> {
874+
return {
875+
resourceSpans: [
876+
{
877+
resource: { attributes: resourceAttributes },
878+
scopeSpans: [
879+
{
880+
scope: { name: "test.instrumentation", version: "1.0.0" },
881+
spans: [
882+
{
883+
traceId,
884+
spanId: "c3Bhbi1pZC0xMjM0NTY=",
885+
name: "test-span",
886+
kind: "SPAN_KIND_INTERNAL" as unknown as ESpanKind,
887+
startTimeUnixNano: "1700000000000000000",
888+
endTimeUnixNano: "1700000001000000000",
889+
attributes: [],
890+
status: { code: "STATUS_CODE_OK" as unknown as EStatusCode },
891+
},
892+
],
893+
},
894+
],
895+
},
896+
],
897+
};
898+
}
899+
900+
describe("Metadata Mapping - resource attributes", () => {
901+
// The opencode OTel plugin (Langy worker) sets tags via OTLP *resource*
902+
// attributes, so reserved keys must be hoisted from the resource the same
903+
// way they are from span attributes. See specs/assistant/langy-otel-tracing.feature.
904+
905+
it("maps tag.tags from resource attributes to labels", async () => {
906+
const request = createOtelTraceWithResourceAttributes([
907+
{ key: "tag.tags", value: { stringValue: "langy" } },
908+
]);
909+
910+
const traces = await openTelemetryTraceRequestToTracesForCollection(request);
911+
expect(traces).toHaveLength(1);
912+
913+
const trace = traces[0]!;
914+
expect(trace.reservedTraceMetadata.labels).toEqual(["langy"]);
915+
});
916+
917+
it("maps langwatch.thread.id from resource attributes to thread_id", async () => {
918+
const request = createOtelTraceWithResourceAttributes([
919+
{ key: "langwatch.thread.id", value: { stringValue: "conv-123" } },
920+
]);
921+
922+
const traces = await openTelemetryTraceRequestToTracesForCollection(request);
923+
expect(traces).toHaveLength(1);
924+
925+
const trace = traces[0]!;
926+
expect(trace.reservedTraceMetadata.thread_id).toBe("conv-123");
927+
});
928+
929+
it("keeps non-reserved resource attributes as custom metadata", async () => {
930+
const request = createOtelTraceWithResourceAttributes([
931+
{ key: "service.name", value: { stringValue: "langy-agent" } },
932+
]);
933+
934+
const traces = await openTelemetryTraceRequestToTracesForCollection(request);
935+
expect(traces).toHaveLength(1);
936+
937+
const trace = traces[0]!;
938+
expect(trace.customMetadata["service.name"]).toBe("langy-agent");
939+
expect(trace.reservedTraceMetadata).not.toHaveProperty("service.name");
940+
});
941+
942+
it("maps the Langy worker's resource attributes end to end", async () => {
943+
// Exactly what services/langy-agent/server.js sets via
944+
// OPENCODE_RESOURCE_ATTRIBUTES for a conversation.
945+
const request = createOtelTraceWithResourceAttributes([
946+
{ key: "tag.tags", value: { stringValue: "langy" } },
947+
{ key: "service.name", value: { stringValue: "langy-agent" } },
948+
{ key: "langwatch.thread.id", value: { stringValue: "conv-123" } },
949+
]);
950+
951+
const traces = await openTelemetryTraceRequestToTracesForCollection(request);
952+
expect(traces).toHaveLength(1);
953+
954+
const trace = traces[0]!;
955+
expect(trace.reservedTraceMetadata.labels).toEqual(["langy"]);
956+
expect(trace.reservedTraceMetadata.thread_id).toBe("conv-123");
957+
expect(trace.customMetadata["service.name"]).toBe("langy-agent");
958+
});
959+
});

‎langwatch/src/server/tracer/otel.traces.ts‎

Lines changed: 45 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -175,11 +175,25 @@ const openTelemetryTraceRequestToTraceForCollection = (
175175
);
176176
const otelTrace = cloneDeep(otelTrace_);
177177

178+
// Collect OTLP resource attributes (shared by every span in the
179+
// request). Reserved keys (tag.tags -> labels, langwatch.thread.id ->
180+
// thread_id, ...) are hoisted to reserved trace metadata exactly as
181+
// they are from span attributes — Langy's opencode OTel plugin sets
182+
// them on the resource, so this is what makes its traces land labeled
183+
// "langy" and grouped by conversation. Everything else stays custom.
178184
const customMetadata: Record<string, any> = {};
185+
const resourceReservedSource: Record<string, any> = {};
179186
for (const resourceSpan of otelTrace.resourceSpans ?? []) {
180187
for (const attribute of resourceSpan?.resource?.attributes ?? []) {
181-
if (attribute?.key) {
182-
customMetadata[attribute.key] = attribute?.value?.stringValue;
188+
if (!attribute?.key) continue;
189+
const value = attribute?.value?.stringValue;
190+
if (
191+
attribute.key in openTelemetryToLangWatchMetadataMapping &&
192+
value != null
193+
) {
194+
resourceReservedSource[attribute.key] = value;
195+
} else {
196+
customMetadata[attribute.key] = value;
183197
}
184198
}
185199
}
@@ -192,6 +206,35 @@ const openTelemetryTraceRequestToTraceForCollection = (
192206
customMetadata,
193207
};
194208

209+
if (Object.keys(resourceReservedSource).length > 0) {
210+
// tag.tags maps to labels (string[]); a resource attribute carries
211+
// it as a single string (OTEL_RESOURCE_ATTRIBUTES can't express
212+
// arrays), so coerce to an array before the reserved schema
213+
// validates it.
214+
if (typeof resourceReservedSource["tag.tags"] === "string") {
215+
resourceReservedSource["tag.tags"] = resourceReservedSource[
216+
"tag.tags"
217+
]
218+
.split(",")
219+
.map((tag: string) => tag.trim())
220+
.filter(Boolean);
221+
}
222+
try {
223+
const { reservedTraceMetadata, customMetadata: extraCustom } =
224+
extractReservedAndCustomMetadata(
225+
applyMappingsToMetadata(resourceReservedSource),
226+
);
227+
trace.reservedTraceMetadata = reservedTraceMetadata;
228+
// Any reserved-source key the schema didn't claim falls back to
229+
// custom metadata rather than being dropped.
230+
Object.assign(trace.customMetadata, extraCustom);
231+
} catch {
232+
// Defensive: never let a malformed resource attribute break
233+
// ingestion — keep the raw values as custom metadata.
234+
Object.assign(trace.customMetadata, resourceReservedSource);
235+
}
236+
}
237+
195238
for (const resourceSpan of otelTrace.resourceSpans ?? []) {
196239
for (const scopeSpan of resourceSpan?.scopeSpans ?? []) {
197240
for (const span of scopeSpan?.spans ?? []) {

‎services/langy-agent/server.js‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,15 @@ const REAPER_INTERVAL_MS = 30_000;
5050
const SESSIONS_ROOT = "/workspace/sessions";
5151
const MAX_BODY_BYTES = 1_000_000; // 1MB — cap /chat body to avoid memory exhaustion.
5252

53+
// opencode has no native OpenTelemetry export, so each worker loads this
54+
// opencode plugin to emit session/llm/tool spans over OTLP. Version is pinned
55+
// by the Dockerfile (OPENCODE_OTEL_PLUGIN_VERSION) so the version opencode
56+
// loads matches the one baked into the image; the literal default keeps the
57+
// manager runnable outside the image.
58+
const OPENCODE_OTEL_PLUGIN = `@devtheops/opencode-plugin-otel@${
59+
process.env.OPENCODE_OTEL_PLUGIN_VERSION || "1.0.0"
60+
}`;
61+
5362
if (!INTERNAL_SECRET) {
5463
console.error("fatal: LANGY_INTERNAL_SECRET is required");
5564
process.exit(1);
@@ -128,6 +137,9 @@ function setupWorkerHome(workerHome, credentials) {
128137
const config = {
129138
$schema: "https://opencode.ai/config.json",
130139
model: "openai/gpt-5-mini",
140+
// OTel plugin: opencode auto-loads it by name and exports spans using the
141+
// OPENCODE_OTLP_* env injected at spawn time (see spawnWorker).
142+
plugin: [OPENCODE_OTEL_PLUGIN],
131143
mcp: {
132144
langwatch: {
133145
type: "local",
@@ -319,6 +331,22 @@ async function spawnWorker(conversationId, credentials) {
319331
OPENAI_API_KEY: credentials.llmVirtualKey,
320332
LANGWATCH_API_KEY: credentials.langwatchApiKey,
321333
LANGWATCH_ENDPOINT: credentials.langwatchEndpoint,
334+
// OTel export (consumed by the opencode OTel plugin, not opencode
335+
// itself). The plugin appends "/v1/traces" to the endpoint, so we point
336+
// it at the "/api/otel" base — LangWatch ingests at /api/otel/v1/traces.
337+
// http/protobuf is what that endpoint accepts; auth is the dedicated
338+
// Langy key as a Bearer token (resolves its own project, no X-Project-Id
339+
// needed). tag.tags=langy becomes the trace label and the conversation
340+
// id groups the chat's turns as one thread. conversationId is charset-
341+
// validated upstream, so it is safe in the comma/= delimited value.
342+
OPENCODE_ENABLE_TELEMETRY: "1",
343+
OPENCODE_OTLP_ENDPOINT: `${credentials.langwatchEndpoint.replace(
344+
/\/+$/,
345+
"",
346+
)}/api/otel`,
347+
OPENCODE_OTLP_PROTOCOL: "http/protobuf",
348+
OPENCODE_OTLP_HEADERS: `Authorization=Bearer ${credentials.langwatchApiKey}`,
349+
OPENCODE_RESOURCE_ATTRIBUTES: `tag.tags=langy,service.name=langy-agent,langwatch.thread.id=${conversationId}`,
322350
},
323351
cwd: workerHome,
324352
stdio: ["ignore", "inherit", "inherit"],
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
@unimplemented
2+
Feature: Langy agent activity is traced into the user's project
3+
As a LangWatch user running Langy
4+
I want Langy's agent activity (reasoning, LLM calls, MCP tool calls) captured as traces
5+
So that I can observe and debug what Langy did, in my own project, tagged "langy"
6+
7+
# Part of epic #4528 (issue #4536).
8+
#
9+
# The Langy worker spawns one `opencode serve` subprocess per conversation
10+
# (services/langy-agent/server.js). opencode has NO native OpenTelemetry
11+
# export, so an opencode telemetry plugin (@devtheops/opencode-plugin-otel,
12+
# pinned) emits session/llm/tool spans over OTLP http/protobuf, authenticated
13+
# with the project's dedicated Langy API key, into LangWatch's existing
14+
# ingest endpoint POST /api/otel/v1/traces. No new REST endpoints.
15+
#
16+
# The plugin tags traces via OTLP *resource* attributes. LangWatch already
17+
# maps reserved keys (tag.tags -> labels, langwatch.thread.id -> thread_id)
18+
# from *span* attributes; this feature also maps them from resource
19+
# attributes so the "langy" label and conversation grouping survive.
20+
21+
Background:
22+
Given a project with a provisioned dedicated Langy API key
23+
And the Langy worker is configured with the opencode OTel plugin
24+
25+
# ============================================================================
26+
# End-to-end: a Langy chat becomes a labeled trace
27+
# ============================================================================
28+
29+
Scenario: A Langy chat produces a trace in the user's project
30+
When I send a message to Langy in my project
31+
Then a trace appears in that same project
32+
And the trace's labels contain "langy"
33+
34+
Scenario: The trace captures the agent's activity as spans
35+
When Langy answers using an LLM and at least one MCP tool
36+
Then the trace contains a span for the LLM call
37+
And the trace contains a span for the MCP tool call
38+
39+
Scenario: Turns of one conversation are grouped together
40+
Given a Langy conversation with id "conv-123"
41+
When I send two messages in that conversation
42+
Then both resulting traces share thread_id "conv-123"
43+
44+
# ============================================================================
45+
# OTLP export wiring (services/langy-agent)
46+
# ============================================================================
47+
48+
Scenario: The worker exports over a protocol LangWatch accepts
49+
Given the worker spawns an opencode subprocess for a conversation
50+
Then the subprocess environment enables the OTel plugin
51+
And the OTLP endpoint resolves to "<LANGWATCH_ENDPOINT>/api/otel/v1/traces"
52+
And the OTLP protocol is "http/protobuf"
53+
And the OTLP headers authenticate with the dedicated Langy API key as a Bearer token
54+
And the OTLP resource attributes include "tag.tags=langy" and the conversation id
55+
56+
Scenario: opencode without the plugin exports nothing
57+
# Guards against the silent no-op: standard OTEL_* env on opencode is ignored.
58+
Given an opencode subprocess without the OTel plugin enabled
59+
When it runs a turn
60+
Then no traces are exported
61+
62+
# ============================================================================
63+
# Ingestion: reserved metadata keys in OTLP resource attributes
64+
# Covered by src/server/tracer/__tests__/metadataLabels.integration.test.ts
65+
# ============================================================================
66+
67+
Scenario: tag.tags in resource attributes becomes trace labels
68+
Given an OTLP trace whose resource attributes include "tag.tags=langy"
69+
When the trace is ingested at /api/otel/v1/traces
70+
Then the stored trace's labels contain "langy"
71+
72+
Scenario: langwatch.thread.id in resource attributes becomes thread_id
73+
Given an OTLP trace whose resource attributes include "langwatch.thread.id=conv-123"
74+
When the trace is ingested
75+
Then the stored trace's thread_id is "conv-123"
76+
77+
Scenario: Non-reserved resource attributes stay as custom metadata
78+
Given an OTLP trace whose resource attributes include "service.name=langy-agent"
79+
When the trace is ingested
80+
Then "service.name" is preserved in the trace's custom metadata
81+
And it is not promoted to a reserved field

0 commit comments

Comments
 (0)