Skip to content

Commit fcaad91

Browse files
Aryansharma28claude
andcommitted
feat(langy): trace opencode worker activity into the user's project (tagged "langy") (#4541)
* feat(langy): trace opencode worker activity into the user's project (tagged "langy") opencode has no native OpenTelemetry export (anomalyco/opencode#12142, #14246, #14697), so the per-session worker now loads the pinned @devtheops/opencode-plugin-otel plugin, which emits session/llm/tool spans over OTLP http/protobuf to LangWatch's existing /api/otel/v1/traces, authenticated with the project's dedicated Langy key (resolves its own project — no X-Project-Id). The plugin tags via OTLP *resource* attributes, but ingestion only mapped the reserved tag.tags->labels / langwatch.thread.id->thread_id keys from *span* attributes. So otel.traces.ts now hoists reserved metadata keys found in resource attributes too (coercing tag.tags to a string[] for labels), additively and guarded so a malformed resource attribute can never break ingestion. This is what makes Langy's traces land labeled "langy" and grouped by conversation. No new REST endpoints (reuses /api/otel/v1/traces). Spec + resource-attribute integration tests added. Closes #4536. Part of epic #4528. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(tracer): keep telemetry.sdk.* resource attrs as custom metadata Hoisting reserved keys from resource attributes also swept the standard telemetry.sdk.* provenance attributes (present on every OTLP trace) into reservedTraceMetadata, changing customMetadata platform-wide and breaking otel.traces.test.ts. Exclude the telemetry.sdk.* family so the change stays scoped to the trace-identity keys Langy needs. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(langy): align langy-route-auth test with post-#4558 gate removal PR #4558 dropped the staff + flag gate from the Langy route, so the non-staff scenario can no longer return 403 with "Langy is not currently enabled" — the route lets the request through and the handler 500s in the no-DB test env. The assertion was stale. Flip the test to anti-regress the removal: pin that NO caller (staff or not, flag on or off) sees the old gate string and isEnabled is never consulted. A re-introduction of the gate would fail loudly. Carries no behavioural change to the OTel work in this PR; this is unblocking the rebase since the test was already broken on the current langy/per-session-manager head. Co-Authored-By: Claude Opus 4.7 <[email protected]> * fix(langy): align ProjectLangyLayout visibility tests with #4558 gate removal useShowLangy stopped reading isLangwatchStaff and the rollout flag in #4558 — Langy now shows for any team member regardless. The two tests flipping gate.staff/flagEnabled and asserting the drawer is null were asserting the pre-#4558 contract and so failed in test-integration (6). Flip both to anti-regress the removal: a team member sees Langy even with the rollout flag off and even when not staff. A re-introduction of either gate would fail loudly. Same pattern as the route-auth fix in 3a15bd9. Co-Authored-By: Claude Opus 4.7 <[email protected]> --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
1 parent 5decf1a commit fcaad91

7 files changed

Lines changed: 319 additions & 56 deletions

File tree

‎Dockerfile.langy_agent‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,11 @@ FROM ubuntu:24.04
1414
# in lockstep with a tested release rather than tracking `latest`.
1515
ARG LANGWATCH_CLI_VERSION=0.31.0
1616
ARG LANGWATCH_MCP_VERSION=0.10.1
17+
# opencode has no native OpenTelemetry export, so the per-session worker loads
18+
# this opencode plugin to emit session/llm/tool spans over OTLP into the user's
19+
# LangWatch project. Pinned; opencode auto-loads it by name from the per-worker
20+
# config (server.js). See specs/assistant/langy-otel-tracing.feature.
21+
ARG OPENCODE_OTEL_PLUGIN_VERSION=1.0.0
1722

1823
RUN apt-get update && apt-get install -y --no-install-recommends \
1924
curl git ca-certificates unzip build-essential python3 \
@@ -32,9 +37,16 @@ RUN curl -fsSL https://opencode.ai/install | bash && \
3237

3338
# CLI (`langwatch`) + MCP server (`langwatch-mcp-server`) from npm. Replaces
3439
# the old locally-`pnpm pack`'d tarballs so the image builds reproducibly in
35-
# CI with no host-side build step.
40+
# CI with no host-side build step. The opencode OTel plugin is installed here
41+
# too so the pinned version is present in the image (and the build fails fast
42+
# if the version is bad) rather than relying solely on opencode's runtime fetch.
3643
RUN npm install -g "langwatch@${LANGWATCH_CLI_VERSION}" \
37-
"@langwatch/mcp-server@${LANGWATCH_MCP_VERSION}"
44+
"@langwatch/mcp-server@${LANGWATCH_MCP_VERSION}" \
45+
"@devtheops/opencode-plugin-otel@${OPENCODE_OTEL_PLUGIN_VERSION}"
46+
47+
# Surface the pinned plugin version to the manager (server.js) so each
48+
# per-worker opencode config requests the exact same version opencode loads.
49+
ENV OPENCODE_OTEL_PLUGIN_VERSION=${OPENCODE_OTEL_PLUGIN_VERSION}
3850

3951
COPY services/langy-agent/entrypoint.sh /entrypoint.sh
4052
COPY services/langy-agent/server.js /server.js

‎langwatch/src/components/langy/__tests__/ProjectLangyLayout.integration.test.tsx‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -152,18 +152,21 @@ describe("ProjectLangyLayout", () => {
152152
});
153153
});
154154

155-
describe("given the visibility gate is not satisfied", () => {
156-
it("does not render Langy when the feature flag is off", () => {
155+
// #4558 dropped the staff + rollout-flag visibility gate; Langy is shown to
156+
// every team member regardless of those inputs. These pin that contract so
157+
// re-introducing either gate would fail loudly.
158+
describe("given the staff / rollout-flag gates were removed in #4558", () => {
159+
it("renders Langy for a team member even when the rollout flag is off", () => {
157160
gate.flagEnabled = false;
158161
renderAt("/demo/traces");
159162
expect(screen.getByText("traces page")).toBeTruthy();
160-
expect(drawer()).toBeNull();
163+
expect(drawer()).not.toBeNull();
161164
});
162165

163-
it("does not render Langy for non-staff users", () => {
166+
it("renders Langy for a non-staff team member", () => {
164167
gate.staff = false;
165168
renderAt("/demo/traces");
166-
expect(drawer()).toBeNull();
169+
expect(drawer()).not.toBeNull();
167170
});
168171
});
169172
});
Lines changed: 39 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,14 @@
11
/**
22
* @vitest-environment node
33
*
4-
* @see specs/assistant/langy-baseline.feature — Access and rollout gating
4+
* Anti-regression for the Langy access-gate removal (#4558): the route used to
5+
* gate non-staff behind `release_langy_enabled`, returning 403 with the literal
6+
* "Langy is not currently enabled" when the flag was off. That gate is gone;
7+
* Langy ships unconditionally to any authenticated session. These tests pin
8+
* that contract so a re-introduction of the staff/flag gate would fail loudly.
59
*
6-
* Binds the Langy access contract:
7-
* - LangWatch staff always reach Langy, even with the rollout flag OFF.
8-
* - Non-staff are blocked unless the rollout flag is on for them.
9-
*
10-
* The flag is the lever for opening Langy beyond staff; it can never lock
11-
* staff out. A future re-introduction of a hard "staff AND flag" gate would
12-
* fail the staff scenario here.
10+
* The downstream handler still 500s in this env (no DB), which is fine — the
11+
* gate is what's under test, not the handler.
1312
*/
1413
import { beforeEach, describe, expect, it, vi } from "vitest";
1514

@@ -25,11 +24,9 @@ vi.mock("~/server/featureFlag", () => ({
2524
},
2625
}));
2726

28-
// The 403 the access middleware emits when neither staff nor rollout lets the
29-
// caller through. Pass-through cases assert the answer is NOT this string —
30-
// downstream handler errors (no DB in this env) are fine; only the gate is
31-
// under test here.
32-
const GATE_BLOCKED = "Langy is not currently enabled";
27+
// The 403 string the old access middleware emitted. Must never reappear in any
28+
// response — its presence means the gate has crept back.
29+
const REMOVED_GATE_MESSAGE = "Langy is not currently enabled";
3330

3431
async function requestLangy() {
3532
const { app } = await import("../langy");
@@ -39,57 +36,52 @@ async function requestLangy() {
3936
);
4037
}
4138

42-
describe("Langy access gating", () => {
39+
describe("Langy access (post-#4558: no staff or rollout gate)", () => {
4340
beforeEach(() => {
4441
vi.clearAllMocks();
4542
});
4643

4744
describe("when the user is LangWatch staff", () => {
48-
describe("and Langy has not been rolled out beyond staff", () => {
49-
it("is not blocked by the rollout gate", async () => {
50-
getServerAuthSession.mockResolvedValue({
51-
user: { email: "[email protected]", id: "u1" },
52-
});
53-
isEnabled.mockResolvedValue(false);
45+
it("reaches the handler regardless of the rollout flag", async () => {
46+
getServerAuthSession.mockResolvedValue({
47+
user: { email: "[email protected]", id: "u1" },
48+
});
49+
isEnabled.mockResolvedValue(false);
5450

55-
const res = await requestLangy();
56-
const body = (await res.json().catch(() => ({}))) as { error?: string };
51+
const res = await requestLangy();
52+
const body = (await res.json().catch(() => ({}))) as { error?: string };
5753

58-
expect(body.error).not.toBe(GATE_BLOCKED);
59-
// Staff must never trigger the flag lookup at all.
60-
expect(isEnabled).not.toHaveBeenCalled();
61-
});
54+
expect(body.error).not.toBe(REMOVED_GATE_MESSAGE);
55+
// The removed gate is the only thing that ever called isEnabled here.
56+
expect(isEnabled).not.toHaveBeenCalled();
6257
});
6358
});
6459

6560
describe("when the user is not staff", () => {
66-
describe("and Langy has not been rolled out to them", () => {
67-
it("is rejected with a 403", async () => {
68-
getServerAuthSession.mockResolvedValue({
69-
user: { email: "[email protected]", id: "u2" },
70-
});
71-
isEnabled.mockResolvedValue(false);
61+
it("reaches the handler with the rollout flag OFF", async () => {
62+
getServerAuthSession.mockResolvedValue({
63+
user: { email: "[email protected]", id: "u2" },
64+
});
65+
isEnabled.mockResolvedValue(false);
7266

73-
const res = await requestLangy();
67+
const res = await requestLangy();
68+
const body = (await res.json().catch(() => ({}))) as { error?: string };
7469

75-
expect(res.status).toBe(403);
76-
const body = (await res.json()) as { error?: string };
77-
expect(body.error).toBe(GATE_BLOCKED);
78-
});
70+
expect(body.error).not.toBe(REMOVED_GATE_MESSAGE);
71+
expect(isEnabled).not.toHaveBeenCalled();
7972
});
8073

81-
describe("and Langy has been rolled out to them", () => {
82-
it("is not blocked by the rollout gate", async () => {
83-
getServerAuthSession.mockResolvedValue({
84-
user: { email: "[email protected]", id: "u3" },
85-
});
86-
isEnabled.mockResolvedValue(true);
74+
it("reaches the handler with the rollout flag ON", async () => {
75+
getServerAuthSession.mockResolvedValue({
76+
user: { email: "[email protected]", id: "u3" },
77+
});
78+
isEnabled.mockResolvedValue(true);
8779

88-
const res = await requestLangy();
89-
const body = (await res.json().catch(() => ({}))) as { error?: string };
80+
const res = await requestLangy();
81+
const body = (await res.json().catch(() => ({}))) as { error?: string };
9082

91-
expect(body.error).not.toBe(GATE_BLOCKED);
92-
});
83+
expect(body.error).not.toBe(REMOVED_GATE_MESSAGE);
84+
expect(isEnabled).not.toHaveBeenCalled();
9385
});
9486
});
9587
});

‎langwatch/src/server/tracer/__tests__/metadataLabels.integration.test.ts‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -859,3 +859,101 @@ describe("Integration - Real-world SDK scenarios", () => {
859859
expect(trace.spans[0]?.type).toBe("llm");
860860
});
861861
});
862+
863+
/**
864+
* Helper to create a minimal OTEL trace request with the given OTLP *resource*
865+
* attributes (as opposed to span attributes). This mirrors how an OTel SDK
866+
* serialises OTEL_RESOURCE_ATTRIBUTES — flat string-valued key/value pairs on
867+
* the resource shared by every span. Used to verify that reserved metadata
868+
* keys are mapped from resource attributes, not only span attributes.
869+
*/
870+
function createOtelTraceWithResourceAttributes(
871+
resourceAttributes: Array<{ key: string; value: { stringValue?: string } }>,
872+
traceId = "dGVzdC10cmFjZS1pZDEyMzQ=",
873+
): DeepPartial<IExportTraceServiceRequest> {
874+
return {
875+
resourceSpans: [
876+
{
877+
resource: { attributes: resourceAttributes },
878+
scopeSpans: [
879+
{
880+
scope: { name: "test.instrumentation", version: "1.0.0" },
881+
spans: [
882+
{
883+
traceId,
884+
spanId: "c3Bhbi1pZC0xMjM0NTY=",
885+
name: "test-span",
886+
kind: "SPAN_KIND_INTERNAL" as unknown as ESpanKind,
887+
startTimeUnixNano: "1700000000000000000",
888+
endTimeUnixNano: "1700000001000000000",
889+
attributes: [],
890+
status: { code: "STATUS_CODE_OK" as unknown as EStatusCode },
891+
},
892+
],
893+
},
894+
],
895+
},
896+
],
897+
};
898+
}
899+
900+
describe("Metadata Mapping - resource attributes", () => {
901+
// The opencode OTel plugin (Langy worker) sets tags via OTLP *resource*
902+
// attributes, so reserved keys must be hoisted from the resource the same
903+
// way they are from span attributes. See specs/assistant/langy-otel-tracing.feature.
904+
905+
it("maps tag.tags from resource attributes to labels", async () => {
906+
const request = createOtelTraceWithResourceAttributes([
907+
{ key: "tag.tags", value: { stringValue: "langy" } },
908+
]);
909+
910+
const traces = await openTelemetryTraceRequestToTracesForCollection(request);
911+
expect(traces).toHaveLength(1);
912+
913+
const trace = traces[0]!;
914+
expect(trace.reservedTraceMetadata.labels).toEqual(["langy"]);
915+
});
916+
917+
it("maps langwatch.thread.id from resource attributes to thread_id", async () => {
918+
const request = createOtelTraceWithResourceAttributes([
919+
{ key: "langwatch.thread.id", value: { stringValue: "conv-123" } },
920+
]);
921+
922+
const traces = await openTelemetryTraceRequestToTracesForCollection(request);
923+
expect(traces).toHaveLength(1);
924+
925+
const trace = traces[0]!;
926+
expect(trace.reservedTraceMetadata.thread_id).toBe("conv-123");
927+
});
928+
929+
it("keeps non-reserved resource attributes as custom metadata", async () => {
930+
const request = createOtelTraceWithResourceAttributes([
931+
{ key: "service.name", value: { stringValue: "langy-agent" } },
932+
]);
933+
934+
const traces = await openTelemetryTraceRequestToTracesForCollection(request);
935+
expect(traces).toHaveLength(1);
936+
937+
const trace = traces[0]!;
938+
expect(trace.customMetadata["service.name"]).toBe("langy-agent");
939+
expect(trace.reservedTraceMetadata).not.toHaveProperty("service.name");
940+
});
941+
942+
it("maps the Langy worker's resource attributes end to end", async () => {
943+
// Exactly what services/langy-agent/server.js sets via
944+
// OPENCODE_RESOURCE_ATTRIBUTES for a conversation.
945+
const request = createOtelTraceWithResourceAttributes([
946+
{ key: "tag.tags", value: { stringValue: "langy" } },
947+
{ key: "service.name", value: { stringValue: "langy-agent" } },
948+
{ key: "langwatch.thread.id", value: { stringValue: "conv-123" } },
949+
]);
950+
951+
const traces = await openTelemetryTraceRequestToTracesForCollection(request);
952+
expect(traces).toHaveLength(1);
953+
954+
const trace = traces[0]!;
955+
expect(trace.reservedTraceMetadata.labels).toEqual(["langy"]);
956+
expect(trace.reservedTraceMetadata.thread_id).toBe("conv-123");
957+
expect(trace.customMetadata["service.name"]).toBe("langy-agent");
958+
});
959+
});

‎langwatch/src/server/tracer/otel.traces.ts‎

Lines changed: 51 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -174,11 +174,31 @@ const openTelemetryTraceRequestToTraceForCollection = (
174174
);
175175
const otelTrace = cloneDeep(otelTrace_);
176176

177+
// Collect OTLP resource attributes (shared by every span in the
178+
// request). Reserved keys (tag.tags -> labels, langwatch.thread.id ->
179+
// thread_id, ...) are hoisted to reserved trace metadata exactly as
180+
// they are from span attributes — Langy's opencode OTel plugin sets
181+
// them on the resource, so this is what makes its traces land labeled
182+
// "langy" and grouped by conversation. Everything else stays custom.
177183
const customMetadata: Record<string, any> = {};
184+
const resourceReservedSource: Record<string, any> = {};
178185
for (const resourceSpan of otelTrace.resourceSpans ?? []) {
179186
for (const attribute of resourceSpan?.resource?.attributes ?? []) {
180-
if (attribute?.key) {
181-
customMetadata[attribute.key] = attribute?.value?.stringValue;
187+
if (!attribute?.key) continue;
188+
const value = attribute?.value?.stringValue;
189+
if (
190+
attribute.key in openTelemetryToLangWatchMetadataMapping &&
191+
// telemetry.sdk.* are standard SDK-provenance resource attributes
192+
// present on every OTLP trace, which LangWatch has always
193+
// surfaced as custom metadata. Leave them there to preserve that
194+
// behaviour and keep this change scoped to the trace-identity
195+
// keys Langy relies on (tag.tags, langwatch.thread.id, ...).
196+
!attribute.key.startsWith("telemetry.sdk.") &&
197+
value != null
198+
) {
199+
resourceReservedSource[attribute.key] = value;
200+
} else {
201+
customMetadata[attribute.key] = value;
182202
}
183203
}
184204
}
@@ -191,6 +211,35 @@ const openTelemetryTraceRequestToTraceForCollection = (
191211
customMetadata,
192212
};
193213

214+
if (Object.keys(resourceReservedSource).length > 0) {
215+
// tag.tags maps to labels (string[]); a resource attribute carries
216+
// it as a single string (OTEL_RESOURCE_ATTRIBUTES can't express
217+
// arrays), so coerce to an array before the reserved schema
218+
// validates it.
219+
if (typeof resourceReservedSource["tag.tags"] === "string") {
220+
resourceReservedSource["tag.tags"] = resourceReservedSource[
221+
"tag.tags"
222+
]
223+
.split(",")
224+
.map((tag: string) => tag.trim())
225+
.filter(Boolean);
226+
}
227+
try {
228+
const { reservedTraceMetadata, customMetadata: extraCustom } =
229+
extractReservedAndCustomMetadata(
230+
applyMappingsToMetadata(resourceReservedSource),
231+
);
232+
trace.reservedTraceMetadata = reservedTraceMetadata;
233+
// Any reserved-source key the schema didn't claim falls back to
234+
// custom metadata rather than being dropped.
235+
Object.assign(trace.customMetadata, extraCustom);
236+
} catch {
237+
// Defensive: never let a malformed resource attribute break
238+
// ingestion — keep the raw values as custom metadata.
239+
Object.assign(trace.customMetadata, resourceReservedSource);
240+
}
241+
}
242+
194243
for (const resourceSpan of otelTrace.resourceSpans ?? []) {
195244
for (const scopeSpan of resourceSpan?.scopeSpans ?? []) {
196245
for (const span of scopeSpan?.spans ?? []) {

0 commit comments

Comments
 (0)