Area(s)

llm, events

Is your change request related to a problem? Please describe.

Prompts and completions are essential part of debugging experience when developing LLM applications.
They could be verbose and contain sensitive information, so they need some opt-in mechanism such as feature-flag, configuration property on the instr library. Specific OTel distros, debugging and diagnostic tools within IDEs may decide to enable the collection by default.

There are multiple ways to capture LLM events: as attributes on spans, as attributes on span events, as log events.

1. Span Attributes:

   • span attributes are limited in size (not on the OTel SDK, but on the vendor side) and could be as small as few kilobytes
   • they are not structured and we'd have to capture the format (e.g. mime type) to allow parsing them
   • but they always stay along with the span making visualization tooling and queries easier. Users can't forward them to a separate backend along with the logs.

2. Span event attributes:

   • span event's don't have a notion of body and are not structured either
   • event attributes are subject to the same problems as span attributes.
   • they are also exported along with the span and arguably are easy for visualization tooling to use
   • their future is uncertain - see Tracking: Clarity and language around stand-alone Events vs. Span Events #695 and SpanEvent and Event through LogRecord should be "unified" opentelemetry-specification#3406

3. Log events/logs:

   • Log events have a notion of body - tracing/logging vendors usually have bigger limits for it than for attributes
   • Log events are structured. We can specify the structure for each event and simplify parsing/querying
   • They are exported separately from spans, but are still correlated with them. If user sends logs and traces to different backends, LLM-debugging experience would need cross-vendor queries which would be challenging.
   • They are not implemented/released/stabilized by most of the languages, eventlogger implementations can only be found in PHP and C++

Since logs/event logs are not ready to be used by instrumentation libraries in popular languages used by LLM applications (python, JS), at least the initial version of LLM semconv cannot use them leaving us with attributes vs span events choice.

Describe the solution you'd like

LLM semantic conventions should allow to capture prompts and completions as span events, but also be future-proof and support new log events:

• I'd like to be able to describe LLM event structure/format in the semantic conventions
• When they are reported as span events, I want to have a designated attribute to capture a body such as event.body (similarly to event.name).
• Same with severity number (see Event yaml definition should allow to document SeverityNumber  #828)

Once/if span events are deprecated, the same conventions could be used to provide back-compat story for the instrumentation libraries or OTel SDKs themselves.

Vendors/exporters may decide to transform such events into log records and would map event.body to the corresponding property with higher length limits than attributes

While these are captured as span events and are sent over the wire along with the spans, vendors/exporters may decide to extract prompts/completions from the event attributes and put them on spans directly to simplify their visualizations and queries.