fix(langfuse_otel): include default tags in OTel v2 - #49858

TLDR

Problem this solves:

• OTel v2 exports caller tags but drops configured defaults

How it solves it:

• Share default-tag expansion with the legacy callback
• Keep caller order, remove duplicates, and tag completed generations

User Flow

Before, the completion succeeds but configured tags are missing from the export

1. Enable OTel v2 and configure default Langfuse tags
2. Send a chat completion with metadata, then repeat with an explicit caller tag
3. Inspect the exported generation and see an empty tag list or just the caller tag

After, the same completions export the configured tags

1. Enable OTel v2 and configure default Langfuse tags
2. Send a chat completion with metadata, then repeat with an explicit caller tag
3. Inspect the exported generation and see the defaults appended after any caller tags

Relevant issues

Fixes #49844

Pre-Submission checklist

• I have added meaningful tests
• The affected test files pass locally
• My PR passes all required CI/CD checks
• The change is scoped to default tags in Langfuse OTel
• Automated reviews have finished and their findings have been addressed

Screenshots / Proof of Fix

Shared setup used llama.cpp b12040 with SmolLM2-135M-Instruct Q4_K_M as the real model provider, Redis 8.10.3 for router and coordination state, and a local OTLP/HTTP receiver. The proxy ran with two workers and OTel v2 enabled. The configured defaults were app and cache_hit. Set LITELLM_OTEL_V2=true and point LANGFUSE_HOST at the local OTLP/HTTP receiver. Configure smollm2 as openai/smollm2 with api_base pointing at llama.cpp, set callbacks: [langfuse_otel], and point both router Redis and coordination Redis at the same instance

python litellm/proxy/proxy_cli.py --config "$QA_CONFIG" --host 127.0.0.1 --port "$QA_PORT" \
  --num_workers 2 --detailed_debug --reload --use_v2_migration_resolver

for tags in '[]' '["caller-tag"]'; do
  curl --fail-with-body -s "http://127.0.0.1:$QA_PORT/v1/chat/completions" \
    -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
    --data "{\"model\":\"smollm2\",\"messages\":[{\"role\":\"user\",\"content\":\"Say hello.\"}],\"max_tokens\":10,\"metadata\":{\"app\":\"local-qa\",\"tags\":$tags}}"
done

Before (ff182c1)

1. Run both requests. Both return HTTP 200 with generated tokens
2. Read langfuse.trace.tags from the generation spans. The values are absent and ["caller-tag"]

After (084be77)

1. Run both requests. Both return HTTP 200 with generated tokens
2. Read langfuse.trace.tags from the generation spans. The values are ["app:local-qa", "cache_hit:None"] and ["caller-tag", "app:local-qa", "cache_hit:None"]

The request span also carries app:local-qa. The cache outcome is added to the completed generation because it is unavailable when the request span is first stamped

The live capture was made before a type-only cleanup of the optional cache key. That cleanup preserved the fallback and passed 187 focused tests. The final commit also passed make check

The four new regression cases fail at the base and pass with this patch. They cover both metadata fields, missing caller tags, duplicate tags, and cache hits and misses

At the final commit, the affected OTel and legacy Langfuse unit suite passed 1,396 tests and the SDK wire suite passed 4 tests