fix(livekit): keep the Future AGI exporter when LiveKit adds its privacy filter (TH-8665) - #273

What does this PR do?

It gets LiveKit voice agents sending real traces again, as traceAI-livekit 0.1.2. Not published; publishing waits for the owner.

1. The exporter is kept. With livekit-agents 1.8, traceAI-livekit sent nothing at all: LiveKit's privacy filter knocked the Future AGI exporter off the provider.
2. The content is sent. Since livekit-agents 1.7, LiveKit names its content attributes lk.pii.*, and the exporter only knew the old names. So spans arrived with no input.value or output.value.
3. A mapping failure is visible. On fi-instrumentation-otel before 1.0.0 the mapping failed on every span and exported unmapped, silently. It now logs one warning and still exports the span. The floor is raised to 1.1.0.
4. The mapping never reaches LiveKit Cloud. The exporter used to write its mapped attributes back onto the shared span. With the lk.pii.* content now mapped, LiveKit Cloud's exporter on the same provider would have received it under names its redaction doesn't strip. The mapping now goes on a copy of each span.

Why?

Linear: TH-8665.

1. The exporter.

• register() marks its own exporter as a default that the next add_span_processor call replaces (fi_instrumentation/otel.py:243-244).
• LiveKit's set_tracer_provider adds its privacy filter through add_span_processor.
• So the exporter was shut down and removed. enable_*_attribute_mapping() now clears that mark before handing the provider to LiveKit. fi_instrumentation and every other framework are unchanged.

2. The content.

• LiveKit renamed 11 content attributes:
  • lk.user_input became lk.pii.user_input;
  • likewise chat_ctx, response.text, response.function_calls, function_tool.arguments, function_tool.output, input_text, user_transcript, instructions, room_name and participant_identity.
• The exporter now reads both names, the newer first.
• LiveKit sets "[]" as the function calls on every LLM node. An empty list no longer counts as output, so output.value is the reply itself. Before, it was {"text": ..., "function_calls": "[]"}, for the older names too.

4. The shared span.

• Every processor on the provider holds the same span object, LiveKit Cloud's exporter among them. LiveKit redacts by key name: it strips keys with a pii segment.
• Our exporter used to write input.value, output.value, metadata, gen_ai.conversation.id and user.id onto that object.
• On 1.8 that reached LiveKit Cloud for spans held until a session turns redaction on; on 1.7, for every content span. The Reviewer reproduced it: Cloud received input.value "my card is 4111".
• The exporter now builds a new ReadableSpan with the mapped attributes, the way LiveKit's own pii.redact() does, and sends that. If anything fails, it sends the original span unmapped, never dropped.
• Also: an empty user input, reply, text to speak, tool argument or tool output no longer becomes the JSON text "".

3. The floor.

• fi-instrumentation-otel 0.1.14 to 0.1.16 lack GEN_AI_SPAN_KIND and GEN_AI_REQUEST_MODEL, so the mapping raised AttributeError, which was swallowed.
• 1.0.0 has both names, but it fails inside register() with OpenTelemetry 1.45, which livekit-agents 1.8.7 and later require. 1.1.0 works.
• So the floor goes from >=0.1.14 to >=1.1.0.
• For customers: between 0.1.x and 1.x, register() only gained one optional argument, so code that just calls register() keeps working. 1.0 moved span attributes from llm.* to the OpenTelemetry GenAI gen_ai.* names. Code that uses those names directly should follow the existing mapping table, which the README links: docs/OTEL_GENAI_SEMANTIC_CONVENTIONS.md, "Migration from OpenInference".

How was it tested?

• 58 tests pass in the package's own setup (poetry install, Python 3.10, livekit-agents 1.8.6, fi-instrumentation-otel 1.1.0 from PyPI). They also pass with livekit-agents 1.8.8 on Python 3.13. On livekit-agents 1.7.0, 49 pass; the 1.8-only integration file skips.
  • Privacy:
    • LiveKit's real gated Cloud exporter sits on the same provider, and job_registered(..., redacted=True) is called. Cloud receives none of input.value, output.value, gen_ai.conversation.id, user.id, and no secret value anywhere. Meanwhile our exporter does get them.
    • On a plain provider, a second exporter sees the span untouched; this also covers 1.7.
    • The copy keeps every other field: resource (project name), parent, events, links, kind, status, times, scope.
  • tests/test_mapping.py: the mapping on both attribute-name sets (LLM, agent turn, tool, TTS, session, metadata); the newer name wins; empty function calls; a mapping failure is logged once with no attribute values and the span is still exported, on both code paths.
  • tests/test_integration.py: the exporter is kept (HTTP and gRPC); spans reach it; a later processor runs next to it; register() on its own is unchanged. There is also an end-to-end turn: a real LiveKit AgentSession turn with a fake model, exported by the real exporter to a local collector, checking the content on the llm_node and agent_turn spans. And the LIVEKIT_TELEMETRY_ALLOW_PII=false opt-out, where content stays out of the export. Each test puts LiveKit's module tracer back afterwards.
• 8 more bugs planted for the privacy fix, all caught: mutating the shared span again; sending the original instead of the copy; dropping a span on failure; the copy losing its resource, parent, status or events; an empty value counted as content. The cold review also planted each renamed-pair swap (9 of 9 caught) and a value leaked into the warning (caught).
• 12 bugs planted by hand, then reverted. All caught:
  • old names only; new names only; name order reversed;
  • warning repeated; warning removed; attributes dropped on failure;
  • empty function calls treated as output; a span dropped on the export path;
  • allow_pii forced on;
  • the exporter fix removed; the fix applied only when something was swapped; register() no longer setting the mark.
• One agent turn, probed end to end through the real exporter (the same output in both setups):

  SPAN llm_node
     gen_ai.request.model = probe-model-1
     gen_ai.span.kind = LLM
     input.value = {"items": [{"id": "lk.agent_task.instructions", "type": "message", "role": "system", ...
     output.value = "Your table for two is booked at 7pm."
  SPAN agent_turn
     gen_ai.span.kind = CHAIN
     input.value = "Book a table for two at 7pm"
     output.value = "Your table for two is booked at 7pm."
  

  Before this PR, the same turn had no input.value or output.value on either span.
• ruff check passes on the changed files.
• The README has a Compatibility section (livekit-agents 1.x with 1.8 tested; fi-instrumentation-otel 1.1.0 or later) and the updated set_tracer_provider note.

Checklist

• Branch is off main
• Commit messages follow Conventional Commits
• No TODOs or commented-out code left in
• No real API keys or secrets (tests use placeholders and a local collector)
• VOCABULARY check (N/A: docs/VOCABULARY.md is not in this repo)

Notes for reviewers

• Dev dependencies: livekit-agents and its two plugins are pinned to 1.8.x (Python 3.10+), and pytest is added. poetry.lock was re-locked with Poetry 2.1.2. The lock's numpy 2.0.2 has no Python 3.13 wheel, so the dev setup needs Python 3.10 to 3.12.
• Intended behaviour changes:
  • A processor added after enable_* runs next to the exporter instead of replacing it.
  • An LLM span with no function calls has the reply alone as its output.
  • A renamed attribute that goes into metadata, such as lk.pii.instructions, keeps the name it arrived with.
  • The copy we send reports zero dropped attributes, events and links. LiveKit's own redact() does the same, and those counts are diagnostic only.
• Not in this PR, found in review:
  • On OpenTelemetry 1.45, enable_* loses headers passed to register(headers=...). This predates the PR; customers using FI_API_KEY/FI_SECRET_KEY are unaffected. Proposed as a follow-up.
  • Some content is still unmapped: the user-turn transcript only goes to metadata, and RPC, SIP and AMD content isn't mapped.
• The underlying register() trap is tracked separately as TH-8672.

🤖 Generated with Claude Code