OpenTelemetry has quickly become the defacto standard for observability, and is being adopted as the observability platform for LLMs.

Distributed tracing enables tracking of complex calls within and between a set of services. Each service collects and transmits its own telemetry to a backend, which can stitch the records together to form the bigger picture based on a couple of identifiers being included in each request. This takes the form of a TraceId which is common for all related activity across a series of requests, and a parent SpanId which is used to form the activity graph. Http, database, LLM requests are typically all modelled as their own spans.

For http, those are a W3C standard header https://www.w3.org/TR/trace-context/.

For the stdio transport, there should be a standard field as part of the protocol for passing the telemetry IDs down. Including the traceparent and tracestate as optional parameters as part of JSON payload will make it much easier to track telemetry across the system.

For http based transports, if there are separate requests for each call, then no additional work is needed. I am not an expert on SSE, but if it streams over a single web request similar to websockets, then there probably needs to be a way to express the TraceIds as part of the JSON RPC payload.