spantruth

Does the telemetry an LLM instrumentation library reports match what the provider actually sent? Each cell replays one provider response through one library and checks every reported value against the wire.

spantruth 0.4.2 · pinned versions · 2026-11-14T10:35:14+00:00 · Python 3.12 · semantic-conventions-genai 6fd0d763 · source

Pages: pinned versions · latest releases · the ten bug classes · what new releases changed. A fix that ships upstream turns its cell green on the latest-releases page first.

pass every graded value matches the wiremissing the span ended without a value the wire carriedwrong a reported value does not match the wirelost the span never ended, so no backend ever receives itbroken the application saw something different from an uninstrumented runinconclusive the cell could not be graded: the harness could not run it, or the library's setup raised

Scenario	anthropic-sdk	ddtrace	langfuse	mlflow	openinference	openlit	openllmetry	otel-official	otel-official-sdk-on	sentry	control-faulty	control-honest
`anthropic/async-basic` One call (async)	missing	missing	·	missing	pass	pass	pass	pass	wrong	pass	pass	pass
`anthropic/async-stream` Stream read to the end (async)	missing	missing	·	missing	pass	pass	wrong	pass	wrong	pass	pass	pass
`anthropic/async-stream-dropped` Connection drops mid-stream (async)	pass	pass	·	wrong	pass	lost	missing	pass	pass	missing	wrong	pass
`anthropic/async-stream-early-exit` App closes the stream after two events (async)	pass	missing	·	missing	pass	lost	lost	pass	wrong	pass	lost	pass
`anthropic/async-stream-error-event` Overloaded error event mid-stream (async)	pass	pass	·	wrong	pass	lost	missing	pass	pass	missing	wrong	pass
`anthropic/basic` One call	missing	missing	·	missing	pass	pass	pass	pass	wrong	pass	pass	pass
`anthropic/cache-hit` Prompt cache read and write	missing	missing	·	missing	pass	wrong	pass	pass	wrong	pass	wrong	pass
`anthropic/rate-limited` HTTP 429	pass	pass	·	missing	pass	missing	pass	pass	pass	missing	wrong	pass
`anthropic/retry-429` HTTP 429 with retry-after, then a success	missing	missing	·	missing	pass	pass	pass	pass	wrong	pass	wrong	pass
`anthropic/stream` Stream read to the end	missing	missing	·	missing	pass	pass	wrong	pass	wrong	pass	pass	pass
`anthropic/stream-cache-cumulative` Cache counts repeated in message_delta	missing	missing	·	missing	pass	wrong	wrong	pass	wrong	pass	wrong	pass
`anthropic/stream-cache-hit` Prompt cache read and write, streamed	missing	missing	·	missing	pass	wrong	wrong	pass	wrong	pass	wrong	pass
`anthropic/stream-dropped` Connection drops mid-stream	pass	pass	·	wrong	pass	lost	lost	pass	pass	missing	wrong	pass
`anthropic/stream-early-exit` App closes the stream after two events	pass	missing	·	missing	pass	lost	lost	pass	wrong	pass	lost	pass
`anthropic/stream-early-exit-with` App leaves the stream after two events with a with block	pass	missing	·	missing	pass	lost	lost	pass	wrong	pass	lost	pass
`anthropic/stream-error-event` Overloaded error event mid-stream	pass	pass	·	wrong	pass	lost	lost	pass	pass	missing	wrong	pass
`anthropic/stream-thinking` Extended thinking, streamed	missing	missing	·	missing	pass	missing	wrong	pass	wrong	missing	pass	pass
`anthropic/stream-tool-call` Two parallel tool uses, streamed	missing	missing	·	missing	pass	pass	wrong	pass	wrong	pass	pass	pass
`anthropic/thinking` Extended thinking	missing	missing	·	missing	pass	missing	missing	pass	wrong	missing	pass	pass
`anthropic/tool-call` Two parallel tool uses	missing	missing	·	missing	pass	pass	pass	pass	wrong	pass	pass	pass
`openai/async-basic` One call (async)	·	wrong	pass	wrong	pass	pass	pass	pass	·	missing	pass	pass
`openai/async-stream` Stream read to the end (async)	·	wrong	pass	wrong	pass	pass	pass	pass	·	missing	pass	pass
`openai/async-stream-dropped` Connection drops mid-stream (async)	·	pass	wrong	missing	pass	lost	wrong	pass	·	lost	wrong	pass
`openai/async-stream-early-exit` App closes the stream after two chunks (async)	·	wrong	pass	wrong	pass	lost	pass	pass	·	lost	lost	pass
`openai/async-stream-helper` Stream via with_streaming_response (async)	·	wrong	missing	missing	missing	lost	missing	pass	·	missing	pass	pass
`openai/basic` One call	·	wrong	pass	wrong	pass	pass	pass	pass	·	missing	pass	pass
`openai/cache-hit` Prompt cache hit	·	wrong	pass	wrong	pass	pass	pass	pass	·	missing	pass	pass
`openai/rate-limited` HTTP 429	·	pass	pass	missing	pass	pass	pass	pass	·	missing	wrong	pass
`openai/reasoning` Reasoning tokens	·	wrong	pass	wrong	pass	missing	pass	pass	·	missing	pass	pass
`openai/responses-stream-missing-finish-abort` Responses API stream cut after eight recorded events	·	pass	wrong	missing	pass	lost	pass	pass	·	lost	wrong	pass
`openai/retry-429` HTTP 429 with retry-after, then a success	·	wrong	pass	wrong	pass	pass	pass	pass	·	missing	wrong	pass
`openai/stream` Stream read to the end	·	wrong	pass	wrong	pass	pass	pass	pass	·	missing	pass	pass
`openai/stream-cache-hit` Prompt cache hit, streamed	·	wrong	pass	wrong	pass	pass	pass	pass	·	missing	pass	pass
`openai/stream-dropped` Connection drops mid-stream	·	pass	wrong	missing	pass	lost	wrong	pass	·	lost	wrong	pass
`openai/stream-early-exit` App closes the stream after two chunks	·	wrong	pass	wrong	pass	lost	pass	pass	·	lost	lost	pass
`openai/stream-early-exit-with` App leaves the stream after two chunks with a with block	·	wrong	pass	wrong	pass	lost	pass	pass	·	lost	lost	pass
`openai/stream-helper` Stream via with_streaming_response	·	wrong	missing	missing	pass	lost	missing	pass	·	missing	pass	pass
`openai/stream-no-usage` Stream without include_usage	·	wrong	pass	wrong	pass	wrong	pass	pass	·	missing	wrong	pass
`openai/stream-reasoning` Reasoning tokens, streamed	·	wrong	pass	wrong	pass	missing	missing	pass	·	missing	pass	pass
`openai/stream-tool-call` Two parallel tool calls, streamed	·	wrong	pass	wrong	pass	pass	pass	pass	·	missing	pass	pass
`openai/tool-call` Two parallel tool calls	·	wrong	pass	wrong	pass	pass	pass	pass	·	missing	pass	pass

The controls run the suite's own reference instrumentation. control-honest has to pass every check, and control-faulty has to fail exactly its planted faults. If either moves, the suite is wrong, not a library. CI checks both, so their cells are not in the findings below.

Bug classes in this run

Bug	Seen in	Largest cost error
ST01 Ghost span The app made the call, and no finished span ever reaches a backend.	29 cells in 4 libraries	-100%
ST02 Double span One call reports its token usage on two spans.	15 cells in 1 library	+100%
ST03 Cache-blind tokens Cached tokens are left out of the input count, or their own counts are wrong.	8 cells in 2 libraries	-100%
ST04 Phantom tokens The span reports more tokens than the provider sent, or invents a count.	8 cells in 3 libraries	+0.2%
ST05 Vanishing tokens The span reports fewer tokens than the provider sent, or none.	17 cells in 5 libraries	-100%
ST06 Hidden reasoning The reasoning tokens that the provider reported do not appear on the span.	15 cells in 5 libraries	—
ST07 False OK A call failed, and its span does not say so, or does not say how.	22 cells in 4 libraries	—
ST09 Silent stop The finish reason is missing or wrong.	69 cells in 6 libraries	—

A cost figure is for the token mix of one scenario, at list price from developers.openai.com and platform.claude.com on 2026-11-06. Input tokens include cache reads and writes, so the uncached part is input minus reads minus writes. A span that never ends reports nothing, and usage on two spans counts twice only in a backend that sums across spans. A backend with another formula shows another number.

Findings

Every finding cites the rule it applies. To dispute one, open an issue: a disputed finding is marked as such in known_issues.toml.

missing anthropic-sdk · anthropic/async-basic · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/async-basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_basic`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--async-basic/v1/messages`
`url.path`	`/s/anthropic--async-basic/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/async-stream · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/async-stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_stream`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.078620811`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--async-stream/v1/messages`
`url.path`	`/s/anthropic--async-stream/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/basic · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_basic`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--basic/v1/messages`
`url.path`	`/s/anthropic--basic/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/cache-hit · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_cache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`4096`
`gen_ai.usage.cache_write.input_tokens`	`519`
`gen_ai.usage.input_tokens`	`5802`
`gen_ai.usage.output_tokens`	`377`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--cache-hit/v1/messages`
`url.path`	`/s/anthropic--cache-hit/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/retry-429 · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/retry-429
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_retry`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`http.request.method`	`POST`
`http.request.resend_count`	`1`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--retry-429/v1/messages`
`url.path`	`/s/anthropic--retry-429/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/stream · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_stream`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.021056148`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream/v1/messages`
`url.path`	`/s/anthropic--stream/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/stream-cache-cumulative · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/stream-cache-cumulative
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_cumul`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.035693874`
`gen_ai.usage.cache_read.input_tokens`	`3583`
`gen_ai.usage.cache_write.input_tokens`	`811`
`gen_ai.usage.input_tokens`	`5673`
`gen_ai.usage.output_tokens`	`467`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-cache-cumulative/v1/messages`
`url.path`	`/s/anthropic--stream-cache-cumulative/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/stream-cache-hit · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/stream-cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_scache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.03191061`
`gen_ai.usage.cache_read.input_tokens`	`6144`
`gen_ai.usage.cache_write.input_tokens`	`737`
`gen_ai.usage.input_tokens`	`8424`
`gen_ai.usage.output_tokens`	`511`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-cache-hit/v1/messages`
`url.path`	`/s/anthropic--stream-cache-hit/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/stream-thinking · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/stream-thinking
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_sthink`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.03530495`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1439`
`gen_ai.usage.reasoning.output_tokens`	`1103`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-thinking/v1/messages`
`url.path`	`/s/anthropic--stream-thinking/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/stream-tool-call · finish_reasonsSilent stop #
expected tool_use, reported nothing

Expected: ['tool_use']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/stream-tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`tool_use`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_stools`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.032750685`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1619`
`gen_ai.usage.output_tokens`	`197`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-tool-call/v1/messages`
`url.path`	`/s/anthropic--stream-tool-call/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/thinking · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/thinking
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_thinking`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`
`gen_ai.usage.reasoning.output_tokens`	`947`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--thinking/v1/messages`
`url.path`	`/s/anthropic--thinking/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing anthropic-sdk · anthropic/tool-call · finish_reasonsSilent stop #
expected tool_use, reported nothing

Expected: ['tool_use']
Reproduce: spantruth run --subject anthropic-sdk --scenario anthropic/tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`tool_use`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_tools`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--tool-call/v1/messages`
`url.path`	`/s/anthropic--tool-call/v1/messages`
`url.scheme`	`http`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/async-basic · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/async-basic
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`9563662610032312978`
`llmobs_trace_id`	`6af83a6300000000690336667429b1ca`
`process_id`	`5518`
`runtime-id`	`d4f9fc3a36114d7e8ac1f62523114278`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/async-basic · response_id #
expected msg_wt_basic, reported nothing

Expected: msg_wt_basic
Reproduce: spantruth run --subject ddtrace --scenario anthropic/async-basic
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`9563662610032312978`
`llmobs_trace_id`	`6af83a6300000000690336667429b1ca`
`process_id`	`5518`
`runtime-id`	`d4f9fc3a36114d7e8ac1f62523114278`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/async-basic · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/async-basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`9563662610032312978`
`llmobs_trace_id`	`6af83a6300000000690336667429b1ca`
`process_id`	`5518`
`runtime-id`	`d4f9fc3a36114d7e8ac1f62523114278`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/async-stream · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/async-stream
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`8931671542171226094`
`llmobs_trace_id`	`6af83a6a00000000f3af5b19941209f8`
`process_id`	`5583`
`runtime-id`	`081261cc98894a56b52ca8c6152f91a0`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/async-stream · response_id #
expected msg_wt_stream, reported nothing

Expected: msg_wt_stream
Reproduce: spantruth run --subject ddtrace --scenario anthropic/async-stream
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`8931671542171226094`
`llmobs_trace_id`	`6af83a6a00000000f3af5b19941209f8`
`process_id`	`5583`
`runtime-id`	`081261cc98894a56b52ca8c6152f91a0`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/async-stream · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/async-stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`8931671542171226094`
`llmobs_trace_id`	`6af83a6a00000000f3af5b19941209f8`
`process_id`	`5583`
`runtime-id`	`081261cc98894a56b52ca8c6152f91a0`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/async-stream-early-exit · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/async-stream-early-exit
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1913`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.span_kind`	`llm`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1913`
`llmobs_parent_id`	`9044406853220572668`
`llmobs_trace_id`	`6af83a71000000001a0cacc5262b456b`
`process_id`	`5655`
`runtime-id`	`253add2c63f04b73bbb2ce13d02f0778`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/async-stream-early-exit · response_id #
expected msg_wt_early, reported nothing

Expected: msg_wt_early
Reproduce: spantruth run --subject ddtrace --scenario anthropic/async-stream-early-exit
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1913`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.span_kind`	`llm`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1913`
`llmobs_parent_id`	`9044406853220572668`
`llmobs_trace_id`	`6af83a71000000001a0cacc5262b456b`
`process_id`	`5655`
`runtime-id`	`253add2c63f04b73bbb2ce13d02f0778`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/basic · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/basic
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`2616610662559907318`
`llmobs_trace_id`	`6af839cb000000005dc30b7d474a735f`
`process_id`	`3743`
`runtime-id`	`3ee82f8f2bda4a7d9c2ac42c60f2205f`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/basic · response_id #
expected msg_wt_basic, reported nothing

Expected: msg_wt_basic
Reproduce: spantruth run --subject ddtrace --scenario anthropic/basic
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`2616610662559907318`
`llmobs_trace_id`	`6af839cb000000005dc30b7d474a735f`
`process_id`	`3743`
`runtime-id`	`3ee82f8f2bda4a7d9c2ac42c60f2205f`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/basic · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`2616610662559907318`
`llmobs_trace_id`	`6af839cb000000005dc30b7d474a735f`
`process_id`	`3743`
`runtime-id`	`3ee82f8f2bda4a7d9c2ac42c60f2205f`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/cache-hit · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/cache-hit
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`4096`
`_dd.llmobs.cache_write_input_tokens`	`519`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5802`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`377`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`6179`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`4096`
`gen_ai.usage.cache_write_input_tokens`	`519`
`gen_ai.usage.input_tokens`	`5802`
`gen_ai.usage.output_tokens`	`377`
`gen_ai.usage.total_tokens`	`6179`
`llmobs_parent_id`	`826802194729025425`
`llmobs_trace_id`	`6af839e100000000262c7509bf73226b`
`process_id`	`4063`
`runtime-id`	`683881782da9471992fc2d57ef4b6f4c`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/cache-hit · response_id #
expected msg_wt_cache, reported nothing

Expected: msg_wt_cache
Reproduce: spantruth run --subject ddtrace --scenario anthropic/cache-hit
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`4096`
`_dd.llmobs.cache_write_input_tokens`	`519`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5802`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`377`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`6179`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`4096`
`gen_ai.usage.cache_write_input_tokens`	`519`
`gen_ai.usage.input_tokens`	`5802`
`gen_ai.usage.output_tokens`	`377`
`gen_ai.usage.total_tokens`	`6179`
`llmobs_parent_id`	`826802194729025425`
`llmobs_trace_id`	`6af839e100000000262c7509bf73226b`
`process_id`	`4063`
`runtime-id`	`683881782da9471992fc2d57ef4b6f4c`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/cache-hit · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`4096`
`_dd.llmobs.cache_write_input_tokens`	`519`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5802`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`377`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`6179`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`4096`
`gen_ai.usage.cache_write_input_tokens`	`519`
`gen_ai.usage.input_tokens`	`5802`
`gen_ai.usage.output_tokens`	`377`
`gen_ai.usage.total_tokens`	`6179`
`llmobs_parent_id`	`826802194729025425`
`llmobs_trace_id`	`6af839e100000000262c7509bf73226b`
`process_id`	`4063`
`runtime-id`	`683881782da9471992fc2d57ef4b6f4c`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/retry-429 · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/retry-429
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2477`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`367`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2844`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`gen_ai.usage.total_tokens`	`2844`
`llmobs_parent_id`	`12769315714654436971`
`llmobs_trace_id`	`6af83a14000000000464d98d35cf1231`
`process_id`	`4700`
`runtime-id`	`b733bcb9fb084a4bb11bb7801d4cca39`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/retry-429 · response_id #
expected msg_wt_retry, reported nothing

Expected: msg_wt_retry
Reproduce: spantruth run --subject ddtrace --scenario anthropic/retry-429
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2477`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`367`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2844`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`gen_ai.usage.total_tokens`	`2844`
`llmobs_parent_id`	`12769315714654436971`
`llmobs_trace_id`	`6af83a14000000000464d98d35cf1231`
`process_id`	`4700`
`runtime-id`	`b733bcb9fb084a4bb11bb7801d4cca39`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/retry-429 · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/retry-429
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2477`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`367`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2844`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`gen_ai.usage.total_tokens`	`2844`
`llmobs_parent_id`	`12769315714654436971`
`llmobs_trace_id`	`6af83a14000000000464d98d35cf1231`
`process_id`	`4700`
`runtime-id`	`b733bcb9fb084a4bb11bb7801d4cca39`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`4223753839333104704`
`llmobs_trace_id`	`6af839d300000000d5dbfebe0e573c83`
`process_id`	`3886`
`runtime-id`	`f3f0d97b29374323bd5ec34d1a138538`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream · response_id #
expected msg_wt_stream, reported nothing

Expected: msg_wt_stream
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`4223753839333104704`
`llmobs_trace_id`	`6af839d300000000d5dbfebe0e573c83`
`process_id`	`3886`
`runtime-id`	`f3f0d97b29374323bd5ec34d1a138538`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`4223753839333104704`
`llmobs_trace_id`	`6af839d300000000d5dbfebe0e573c83`
`process_id`	`3886`
`runtime-id`	`f3f0d97b29374323bd5ec34d1a138538`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-cache-cumulative · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-cache-cumulative
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`3583`
`_dd.llmobs.cache_write_input_tokens`	`811`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5673`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`467`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`6140`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`3583`
`gen_ai.usage.cache_write_input_tokens`	`811`
`gen_ai.usage.input_tokens`	`5673`
`gen_ai.usage.output_tokens`	`467`
`gen_ai.usage.total_tokens`	`6140`
`llmobs_parent_id`	`4828640898477137592`
`llmobs_trace_id`	`6af839f000000000b13927d27ac19942`
`process_id`	`4193`
`runtime-id`	`0cca06f7d8af4def92456e380774d20c`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-cache-cumulative · response_id #
expected msg_wt_cumul, reported nothing

Expected: msg_wt_cumul
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-cache-cumulative
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`3583`
`_dd.llmobs.cache_write_input_tokens`	`811`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5673`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`467`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`6140`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`3583`
`gen_ai.usage.cache_write_input_tokens`	`811`
`gen_ai.usage.input_tokens`	`5673`
`gen_ai.usage.output_tokens`	`467`
`gen_ai.usage.total_tokens`	`6140`
`llmobs_parent_id`	`4828640898477137592`
`llmobs_trace_id`	`6af839f000000000b13927d27ac19942`
`process_id`	`4193`
`runtime-id`	`0cca06f7d8af4def92456e380774d20c`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-cache-cumulative · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-cache-cumulative
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`3583`
`_dd.llmobs.cache_write_input_tokens`	`811`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5673`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`467`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`6140`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`3583`
`gen_ai.usage.cache_write_input_tokens`	`811`
`gen_ai.usage.input_tokens`	`5673`
`gen_ai.usage.output_tokens`	`467`
`gen_ai.usage.total_tokens`	`6140`
`llmobs_parent_id`	`4828640898477137592`
`llmobs_trace_id`	`6af839f000000000b13927d27ac19942`
`process_id`	`4193`
`runtime-id`	`0cca06f7d8af4def92456e380774d20c`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-cache-hit · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-cache-hit
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`6144`
`_dd.llmobs.cache_write_input_tokens`	`737`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`8424`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`511`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`8935`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`6144`
`gen_ai.usage.cache_write_input_tokens`	`737`
`gen_ai.usage.input_tokens`	`8424`
`gen_ai.usage.output_tokens`	`511`
`gen_ai.usage.total_tokens`	`8935`
`llmobs_parent_id`	`17405815657294882575`
`llmobs_trace_id`	`6af839e9000000005b662475d82f640b`
`process_id`	`4120`
`runtime-id`	`f7cdb86b5a124c6885dc60b59d95fc32`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-cache-hit · response_id #
expected msg_wt_scache, reported nothing

Expected: msg_wt_scache
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-cache-hit
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`6144`
`_dd.llmobs.cache_write_input_tokens`	`737`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`8424`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`511`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`8935`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`6144`
`gen_ai.usage.cache_write_input_tokens`	`737`
`gen_ai.usage.input_tokens`	`8424`
`gen_ai.usage.output_tokens`	`511`
`gen_ai.usage.total_tokens`	`8935`
`llmobs_parent_id`	`17405815657294882575`
`llmobs_trace_id`	`6af839e9000000005b662475d82f640b`
`process_id`	`4120`
`runtime-id`	`f7cdb86b5a124c6885dc60b59d95fc32`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-cache-hit · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`6144`
`_dd.llmobs.cache_write_input_tokens`	`737`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`8424`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`511`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`8935`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`6144`
`gen_ai.usage.cache_write_input_tokens`	`737`
`gen_ai.usage.input_tokens`	`8424`
`gen_ai.usage.output_tokens`	`511`
`gen_ai.usage.total_tokens`	`8935`
`llmobs_parent_id`	`17405815657294882575`
`llmobs_trace_id`	`6af839e9000000005b662475d82f640b`
`process_id`	`4120`
`runtime-id`	`f7cdb86b5a124c6885dc60b59d95fc32`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-early-exit · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-early-exit
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1913`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.span_kind`	`llm`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1913`
`llmobs_parent_id`	`2536880817615418833`
`llmobs_trace_id`	`6af839da000000005903ca50f57a0c4e`
`process_id`	`3995`
`runtime-id`	`7087442e211d46e19c8b05ccc0947d90`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-early-exit · response_id #
expected msg_wt_early, reported nothing

Expected: msg_wt_early
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-early-exit
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1913`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.span_kind`	`llm`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1913`
`llmobs_parent_id`	`2536880817615418833`
`llmobs_trace_id`	`6af839da000000005903ca50f57a0c4e`
`process_id`	`3995`
`runtime-id`	`7087442e211d46e19c8b05ccc0947d90`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-early-exit-with · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-early-exit-with
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1913`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.span_kind`	`llm`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1913`
`llmobs_parent_id`	`14686687379566588714`
`llmobs_trace_id`	`6af83a3800000000eb663eb4d1a9bc6d`
`process_id`	`5053`
`runtime-id`	`bdc93dfc8a354c1e8e88cf716ec45848`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-early-exit-with · response_id #
expected msg_wt_early, reported nothing

Expected: msg_wt_early
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-early-exit-with
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1913`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.span_kind`	`llm`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1913`
`llmobs_parent_id`	`14686687379566588714`
`llmobs_trace_id`	`6af83a3800000000eb663eb4d1a9bc6d`
`process_id`	`5053`
`runtime-id`	`bdc93dfc8a354c1e8e88cf716ec45848`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-thinking · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-thinking
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1831`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`1439`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`3270`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1439`
`gen_ai.usage.total_tokens`	`3270`
`llmobs_parent_id`	`17066759561132326688`
`llmobs_trace_id`	`6af83a2a00000000a1c1d78dc415de12`
`process_id`	`4914`
`runtime-id`	`0bc652574e4d49cd98c80381978f41c9`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-thinking · response_id #
expected msg_wt_sthink, reported nothing

Expected: msg_wt_sthink
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-thinking
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1831`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`1439`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`3270`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1439`
`gen_ai.usage.total_tokens`	`3270`
`llmobs_parent_id`	`17066759561132326688`
`llmobs_trace_id`	`6af83a2a00000000a1c1d78dc415de12`
`process_id`	`4914`
`runtime-id`	`0bc652574e4d49cd98c80381978f41c9`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-thinking · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-thinking
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1831`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`1439`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`3270`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1439`
`gen_ai.usage.total_tokens`	`3270`
`llmobs_parent_id`	`17066759561132326688`
`llmobs_trace_id`	`6af83a2a00000000a1c1d78dc415de12`
`process_id`	`4914`
`runtime-id`	`0bc652574e4d49cd98c80381978f41c9`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-thinking · reasoning_tokensHidden reasoning #
expected 1103, reported nothing

Expected: 1103
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1831`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`1439`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`3270`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1439`
`gen_ai.usage.total_tokens`	`3270`
`llmobs_parent_id`	`17066759561132326688`
`llmobs_trace_id`	`6af83a2a00000000a1c1d78dc415de12`
`process_id`	`4914`
`runtime-id`	`0bc652574e4d49cd98c80381978f41c9`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-tool-call · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-tool-call
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1619`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`197`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1816`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1619`
`gen_ai.usage.output_tokens`	`197`
`gen_ai.usage.total_tokens`	`1816`
`llmobs_parent_id`	`971931045457124499`
`llmobs_trace_id`	`6af83a0600000000805c7b52b3cd5400`
`process_id`	`4515`
`runtime-id`	`e93c34a212ad4578a3d89b5eb9b37491`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-tool-call · response_id #
expected msg_wt_stools, reported nothing

Expected: msg_wt_stools
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-tool-call
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1619`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`197`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1816`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1619`
`gen_ai.usage.output_tokens`	`197`
`gen_ai.usage.total_tokens`	`1816`
`llmobs_parent_id`	`971931045457124499`
`llmobs_trace_id`	`6af83a0600000000805c7b52b3cd5400`
`process_id`	`4515`
`runtime-id`	`e93c34a212ad4578a3d89b5eb9b37491`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/stream-tool-call · finish_reasonsSilent stop #
expected tool_use, reported nothing

Expected: ['tool_use']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/stream-tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1619`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`197`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1816`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1619`
`gen_ai.usage.output_tokens`	`197`
`gen_ai.usage.total_tokens`	`1816`
`llmobs_parent_id`	`971931045457124499`
`llmobs_trace_id`	`6af83a0600000000805c7b52b3cd5400`
`process_id`	`4515`
`runtime-id`	`e93c34a212ad4578a3d89b5eb9b37491`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/thinking · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/thinking
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1627`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`1273`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2900`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`
`gen_ai.usage.total_tokens`	`2900`
`llmobs_parent_id`	`16259453098778130409`
`llmobs_trace_id`	`6af83a2300000000737a04ddbc6e6c5d`
`process_id`	`4851`
`runtime-id`	`18cd679c06ec4356ba8b0b252407d977`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/thinking · response_id #
expected msg_wt_thinking, reported nothing

Expected: msg_wt_thinking
Reproduce: spantruth run --subject ddtrace --scenario anthropic/thinking
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1627`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`1273`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2900`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`
`gen_ai.usage.total_tokens`	`2900`
`llmobs_parent_id`	`16259453098778130409`
`llmobs_trace_id`	`6af83a2300000000737a04ddbc6e6c5d`
`process_id`	`4851`
`runtime-id`	`18cd679c06ec4356ba8b0b252407d977`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/thinking · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/thinking
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1627`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`1273`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2900`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`
`gen_ai.usage.total_tokens`	`2900`
`llmobs_parent_id`	`16259453098778130409`
`llmobs_trace_id`	`6af83a2300000000737a04ddbc6e6c5d`
`process_id`	`4851`
`runtime-id`	`18cd679c06ec4356ba8b0b252407d977`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/thinking · reasoning_tokensHidden reasoning #
expected 947, reported nothing

Expected: 947
Reproduce: spantruth run --subject ddtrace --scenario anthropic/thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1627`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`1273`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2900`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`
`gen_ai.usage.total_tokens`	`2900`
`llmobs_parent_id`	`16259453098778130409`
`llmobs_trace_id`	`6af83a2300000000737a04ddbc6e6c5d`
`process_id`	`4851`
`runtime-id`	`18cd679c06ec4356ba8b0b252407d977`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/tool-call · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject ddtrace --scenario anthropic/tool-call
Rule: The name of the model that generated the response.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`931`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`88`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1019`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`gen_ai.usage.total_tokens`	`1019`
`llmobs_parent_id`	`15328357555578350988`
`llmobs_trace_id`	`6af839fe000000006b771b2be67dad8b`
`process_id`	`4319`
`runtime-id`	`abaf0ee9f80341e0a8218c6c5c12bbc8`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/tool-call · response_id #
expected msg_wt_tools, reported nothing

Expected: msg_wt_tools
Reproduce: spantruth run --subject ddtrace --scenario anthropic/tool-call
Rule: The unique identifier of the completion.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`931`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`88`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1019`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`gen_ai.usage.total_tokens`	`1019`
`llmobs_parent_id`	`15328357555578350988`
`llmobs_trace_id`	`6af839fe000000006b771b2be67dad8b`
`process_id`	`4319`
`runtime-id`	`abaf0ee9f80341e0a8218c6c5c12bbc8`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · anthropic/tool-call · finish_reasonsSilent stop #
expected tool_use, reported nothing

Expected: ['tool_use']
Reproduce: spantruth run --subject ddtrace --scenario anthropic/tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

anthropic.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.cache_write_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`931`
`_dd.llmobs.model_name`	`claude-sonnet-5-5`
`_dd.llmobs.model_provider`	`anthropic`
`_dd.llmobs.output_tokens`	`88`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1019`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`anthropic.request.model`	`claude-sonnet-5-5`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.cache_write_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`gen_ai.usage.total_tokens`	`1019`
`llmobs_parent_id`	`15328357555578350988`
`llmobs_trace_id`	`6af839fe000000006b771b2be67dad8b`
`process_id`	`4319`
`runtime-id`	`abaf0ee9f80341e0a8218c6c5c12bbc8`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/async-basic · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/async-basic
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`3957638062865037862`
`llmobs_trace_id`	`6af83a3f000000008bf1b2e4e431e170`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5115`
`runtime-id`	`58c4281b99264f5e96db44072e4b3da2`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-basic · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/async-basic
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`3957638062865037862`
`llmobs_trace_id`	`6af83a3f000000008bf1b2e4e431e170`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5115`
`runtime-id`	`58c4281b99264f5e96db44072e4b3da2`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-basic · response_id #
expected chatcmpl-wt-basic, reported nothing

Expected: chatcmpl-wt-basic
Reproduce: spantruth run --subject ddtrace --scenario openai/async-basic
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`3957638062865037862`
`llmobs_trace_id`	`6af83a3f000000008bf1b2e4e431e170`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5115`
`runtime-id`	`58c4281b99264f5e96db44072e4b3da2`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-basic · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/async-basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`3957638062865037862`
`llmobs_trace_id`	`6af83a3f000000008bf1b2e4e431e170`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5115`
`runtime-id`	`58c4281b99264f5e96db44072e4b3da2`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/async-stream · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`17669205400087717522`
`llmobs_trace_id`	`6af83a46000000004e59994030b4e1f0`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5176`
`runtime-id`	`d29f5c52c9724fb38fba5800cec20a87`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-stream · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`17669205400087717522`
`llmobs_trace_id`	`6af83a46000000004e59994030b4e1f0`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5176`
`runtime-id`	`d29f5c52c9724fb38fba5800cec20a87`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-stream · response_id #
expected chatcmpl-wt-stream, reported nothing

Expected: chatcmpl-wt-stream
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`17669205400087717522`
`llmobs_trace_id`	`6af83a46000000004e59994030b4e1f0`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5176`
`runtime-id`	`d29f5c52c9724fb38fba5800cec20a87`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-stream · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`17669205400087717522`
`llmobs_trace_id`	`6af83a46000000004e59994030b4e1f0`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5176`
`runtime-id`	`d29f5c52c9724fb38fba5800cec20a87`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/async-stream-early-exit · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream-early-exit
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`9`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`2`
`gen_ai.usage.total_tokens`	`9`
`llmobs_parent_id`	`1948899735922544329`
`llmobs_trace_id`	`6af83a4d000000005e967acdaa6d72b8`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5328`
`runtime-id`	`37946e435d40422e8ca5878b004e0d9e`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-stream-early-exit · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream-early-exit
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`9`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`2`
`gen_ai.usage.total_tokens`	`9`
`llmobs_parent_id`	`1948899735922544329`
`llmobs_trace_id`	`6af83a4d000000005e967acdaa6d72b8`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5328`
`runtime-id`	`37946e435d40422e8ca5878b004e0d9e`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-stream-early-exit · response_id #
expected chatcmpl-wt-early, reported nothing

Expected: chatcmpl-wt-early
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream-early-exit
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`9`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`2`
`gen_ai.usage.total_tokens`	`9`
`llmobs_parent_id`	`1948899735922544329`
`llmobs_trace_id`	`6af83a4d000000005e967acdaa6d72b8`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5328`
`runtime-id`	`37946e435d40422e8ca5878b004e0d9e`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/async-stream-helper · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream-helper
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2137`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`313`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2137`
`gen_ai.usage.output_tokens`	`313`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2450`
`llmobs_parent_id`	`16737060443083423150`
`llmobs_trace_id`	`6af83a5b00000000cb6d543c715c849c`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5455`
`runtime-id`	`03c7052c2412440e8279419eb5f6df24`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-stream-helper · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream-helper
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2137`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`313`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2137`
`gen_ai.usage.output_tokens`	`313`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2450`
`llmobs_parent_id`	`16737060443083423150`
`llmobs_trace_id`	`6af83a5b00000000cb6d543c715c849c`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5455`
`runtime-id`	`03c7052c2412440e8279419eb5f6df24`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-stream-helper · response_id #
expected chatcmpl-wt-shelper, reported nothing

Expected: chatcmpl-wt-shelper
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream-helper
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2137`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`313`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2137`
`gen_ai.usage.output_tokens`	`313`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2450`
`llmobs_parent_id`	`16737060443083423150`
`llmobs_trace_id`	`6af83a5b00000000cb6d543c715c849c`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5455`
`runtime-id`	`03c7052c2412440e8279419eb5f6df24`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/async-stream-helper · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/async-stream-helper
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2137`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`313`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2137`
`gen_ai.usage.output_tokens`	`313`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2450`
`llmobs_parent_id`	`16737060443083423150`
`llmobs_trace_id`	`6af83a5b00000000cb6d543c715c849c`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`5455`
`runtime-id`	`03c7052c2412440e8279419eb5f6df24`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/basic · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/basic
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`12737379511284336200`
`llmobs_trace_id`	`6af8396a000000008c5b3cf2aaf746ea`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2732`
`runtime-id`	`ba5924cda9bf48ef945f9f97066cf466`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/basic · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/basic
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`12737379511284336200`
`llmobs_trace_id`	`6af8396a000000008c5b3cf2aaf746ea`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2732`
`runtime-id`	`ba5924cda9bf48ef945f9f97066cf466`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/basic · response_id #
expected chatcmpl-wt-basic, reported nothing

Expected: chatcmpl-wt-basic
Reproduce: spantruth run --subject ddtrace --scenario openai/basic
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`12737379511284336200`
`llmobs_trace_id`	`6af8396a000000008c5b3cf2aaf746ea`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2732`
`runtime-id`	`ba5924cda9bf48ef945f9f97066cf466`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/basic · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1187`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`263`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1450`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1450`
`llmobs_parent_id`	`12737379511284336200`
`llmobs_trace_id`	`6af8396a000000008c5b3cf2aaf746ea`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2732`
`runtime-id`	`ba5924cda9bf48ef945f9f97066cf466`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/cache-hit · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/cache-hit
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`4096`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5379`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`377`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`5756`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`4096`
`gen_ai.usage.input_tokens`	`5379`
`gen_ai.usage.output_tokens`	`377`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`5756`
`llmobs_parent_id`	`8387526728005252836`
`llmobs_trace_id`	`6af8397f000000002108df9d9e71cd02`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2914`
`runtime-id`	`b9e6328f2ce049f398d8dbd079d9ccc1`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/cache-hit · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/cache-hit
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`4096`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5379`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`377`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`5756`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`4096`
`gen_ai.usage.input_tokens`	`5379`
`gen_ai.usage.output_tokens`	`377`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`5756`
`llmobs_parent_id`	`8387526728005252836`
`llmobs_trace_id`	`6af8397f000000002108df9d9e71cd02`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2914`
`runtime-id`	`b9e6328f2ce049f398d8dbd079d9ccc1`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/cache-hit · response_id #
expected chatcmpl-wt-cache, reported nothing

Expected: chatcmpl-wt-cache
Reproduce: spantruth run --subject ddtrace --scenario openai/cache-hit
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`4096`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5379`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`377`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`5756`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`4096`
`gen_ai.usage.input_tokens`	`5379`
`gen_ai.usage.output_tokens`	`377`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`5756`
`llmobs_parent_id`	`8387526728005252836`
`llmobs_trace_id`	`6af8397f000000002108df9d9e71cd02`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2914`
`runtime-id`	`b9e6328f2ce049f398d8dbd079d9ccc1`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/cache-hit · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`4096`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`5379`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`377`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`5756`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`4096`
`gen_ai.usage.input_tokens`	`5379`
`gen_ai.usage.output_tokens`	`377`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`5756`
`llmobs_parent_id`	`8387526728005252836`
`llmobs_trace_id`	`6af8397f000000002108df9d9e71cd02`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2914`
`runtime-id`	`b9e6328f2ce049f398d8dbd079d9ccc1`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/reasoning · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/reasoning
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1601`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2219`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`3820`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1601`
`gen_ai.usage.output_tokens`	`2219`
`gen_ai.usage.reasoning_output_tokens`	`1792`
`gen_ai.usage.total_tokens`	`3820`
`llmobs_parent_id`	`10935272085933484972`
`llmobs_trace_id`	`6af839b7000000008e1cbab660253a3e`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3497`
`runtime-id`	`c2a047cfe5d74743a5ab754ce60379b7`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/reasoning · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/reasoning
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1601`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2219`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`3820`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1601`
`gen_ai.usage.output_tokens`	`2219`
`gen_ai.usage.reasoning_output_tokens`	`1792`
`gen_ai.usage.total_tokens`	`3820`
`llmobs_parent_id`	`10935272085933484972`
`llmobs_trace_id`	`6af839b7000000008e1cbab660253a3e`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3497`
`runtime-id`	`c2a047cfe5d74743a5ab754ce60379b7`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/reasoning · response_id #
expected chatcmpl-wt-reason, reported nothing

Expected: chatcmpl-wt-reason
Reproduce: spantruth run --subject ddtrace --scenario openai/reasoning
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1601`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2219`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`3820`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1601`
`gen_ai.usage.output_tokens`	`2219`
`gen_ai.usage.reasoning_output_tokens`	`1792`
`gen_ai.usage.total_tokens`	`3820`
`llmobs_parent_id`	`10935272085933484972`
`llmobs_trace_id`	`6af839b7000000008e1cbab660253a3e`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3497`
`runtime-id`	`c2a047cfe5d74743a5ab754ce60379b7`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/reasoning · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/reasoning
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1601`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2219`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`3820`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1601`
`gen_ai.usage.output_tokens`	`2219`
`gen_ai.usage.reasoning_output_tokens`	`1792`
`gen_ai.usage.total_tokens`	`3820`
`llmobs_parent_id`	`10935272085933484972`
`llmobs_trace_id`	`6af839b7000000008e1cbab660253a3e`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3497`
`runtime-id`	`c2a047cfe5d74743a5ab754ce60379b7`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/retry-429 · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/retry-429
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2477`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`367`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2844`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2844`
`llmobs_parent_id`	`10706526680253147057`
`llmobs_trace_id`	`6af839a9000000007354706c9b775981`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3347`
`runtime-id`	`f211d68969f34ea78727acbcbc21a384`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/retry-429 · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/retry-429
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2477`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`367`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2844`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2844`
`llmobs_parent_id`	`10706526680253147057`
`llmobs_trace_id`	`6af839a9000000007354706c9b775981`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3347`
`runtime-id`	`f211d68969f34ea78727acbcbc21a384`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/retry-429 · response_id #
expected chatcmpl-wt-retry, reported nothing

Expected: chatcmpl-wt-retry
Reproduce: spantruth run --subject ddtrace --scenario openai/retry-429
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2477`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`367`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2844`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2844`
`llmobs_parent_id`	`10706526680253147057`
`llmobs_trace_id`	`6af839a9000000007354706c9b775981`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3347`
`runtime-id`	`f211d68969f34ea78727acbcbc21a384`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/retry-429 · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/retry-429
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2477`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`367`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2844`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2844`
`llmobs_parent_id`	`10706526680253147057`
`llmobs_trace_id`	`6af839a9000000007354706c9b775981`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3347`
`runtime-id`	`f211d68969f34ea78727acbcbc21a384`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/stream
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`14417992398180592432`
`llmobs_trace_id`	`6af83971000000007186c779dad23e2a`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2798`
`runtime-id`	`dfa580650f664e4d941a05524a67b150`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/stream
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`14417992398180592432`
`llmobs_trace_id`	`6af83971000000007186c779dad23e2a`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2798`
`runtime-id`	`dfa580650f664e4d941a05524a67b150`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream · response_id #
expected chatcmpl-wt-stream, reported nothing

Expected: chatcmpl-wt-stream
Reproduce: spantruth run --subject ddtrace --scenario openai/stream
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`14417992398180592432`
`llmobs_trace_id`	`6af83971000000007186c779dad23e2a`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2798`
`runtime-id`	`dfa580650f664e4d941a05524a67b150`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2311`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`419`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`2730`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`2730`
`llmobs_parent_id`	`14417992398180592432`
`llmobs_trace_id`	`6af83971000000007186c779dad23e2a`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2798`
`runtime-id`	`dfa580650f664e4d941a05524a67b150`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-cache-hit · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-cache-hit
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`6144`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7213`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`511`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`7724`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`6144`
`gen_ai.usage.input_tokens`	`7213`
`gen_ai.usage.output_tokens`	`511`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`7724`
`llmobs_parent_id`	`12767192668278124216`
`llmobs_trace_id`	`6af83986000000007f31501dff7a23ae`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2969`
`runtime-id`	`8a6d356161674b5787bf97318f2da232`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-cache-hit · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-cache-hit
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`6144`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7213`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`511`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`7724`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`6144`
`gen_ai.usage.input_tokens`	`7213`
`gen_ai.usage.output_tokens`	`511`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`7724`
`llmobs_parent_id`	`12767192668278124216`
`llmobs_trace_id`	`6af83986000000007f31501dff7a23ae`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2969`
`runtime-id`	`8a6d356161674b5787bf97318f2da232`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-cache-hit · response_id #
expected chatcmpl-wt-scache, reported nothing

Expected: chatcmpl-wt-scache
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-cache-hit
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`6144`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7213`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`511`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`7724`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`6144`
`gen_ai.usage.input_tokens`	`7213`
`gen_ai.usage.output_tokens`	`511`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`7724`
`llmobs_parent_id`	`12767192668278124216`
`llmobs_trace_id`	`6af83986000000007f31501dff7a23ae`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2969`
`runtime-id`	`8a6d356161674b5787bf97318f2da232`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-cache-hit · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`6144`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7213`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`511`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`7724`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`6144`
`gen_ai.usage.input_tokens`	`7213`
`gen_ai.usage.output_tokens`	`511`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`7724`
`llmobs_parent_id`	`12767192668278124216`
`llmobs_trace_id`	`6af83986000000007f31501dff7a23ae`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2969`
`runtime-id`	`8a6d356161674b5787bf97318f2da232`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-early-exit · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-early-exit
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`9`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`2`
`gen_ai.usage.total_tokens`	`9`
`llmobs_parent_id`	`1913079584775218040`
`llmobs_trace_id`	`6af8397800000000a062d05acab65bef`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2856`
`runtime-id`	`8b26d941a03a4c83b0f1068d7851d919`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-early-exit · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-early-exit
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`9`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`2`
`gen_ai.usage.total_tokens`	`9`
`llmobs_parent_id`	`1913079584775218040`
`llmobs_trace_id`	`6af8397800000000a062d05acab65bef`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2856`
`runtime-id`	`8b26d941a03a4c83b0f1068d7851d919`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-early-exit · response_id #
expected chatcmpl-wt-early, reported nothing

Expected: chatcmpl-wt-early
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-early-exit
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`9`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`2`
`gen_ai.usage.total_tokens`	`9`
`llmobs_parent_id`	`1913079584775218040`
`llmobs_trace_id`	`6af8397800000000a062d05acab65bef`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`2856`
`runtime-id`	`8b26d941a03a4c83b0f1068d7851d919`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-early-exit-with · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-early-exit-with
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`9`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`2`
`gen_ai.usage.total_tokens`	`9`
`llmobs_parent_id`	`2957451587458858406`
`llmobs_trace_id`	`6af83a3100000000ae28ceadb9089373`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`4977`
`runtime-id`	`9c9ebfce5e1d4e8daeda7198f9d1f8b8`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-early-exit-with · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-early-exit-with
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`9`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`2`
`gen_ai.usage.total_tokens`	`9`
`llmobs_parent_id`	`2957451587458858406`
`llmobs_trace_id`	`6af83a3100000000ae28ceadb9089373`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`4977`
`runtime-id`	`9c9ebfce5e1d4e8daeda7198f9d1f8b8`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-early-exit-with · response_id #
expected chatcmpl-wt-early, reported nothing

Expected: chatcmpl-wt-early
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-early-exit-with
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`9`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`2`
`gen_ai.usage.total_tokens`	`9`
`llmobs_parent_id`	`2957451587458858406`
`llmobs_trace_id`	`6af83a3100000000ae28ceadb9089373`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`4977`
`runtime-id`	`9c9ebfce5e1d4e8daeda7198f9d1f8b8`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-helper · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Cost: $0.00116 on the wire, $1.75e-06 from the spans (-99.8%), at list price.
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-helper
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`0`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`7`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`0`
`gen_ai.usage.total_tokens`	`7`
`llmobs_parent_id`	`18279313228410085241`
`llmobs_trace_id`	`6af839c400000000a027947968c2eb53`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`process_id`	`3656`
`runtime-id`	`f441501157654b638091afc07b8ec261`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-helper · response_id #
expected chatcmpl-wt-shelper, reported nothing

Expected: chatcmpl-wt-shelper
Cost: $0.00116 on the wire, $1.75e-06 from the spans (-99.8%), at list price.
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-helper
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`0`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`7`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`0`
`gen_ai.usage.total_tokens`	`7`
`llmobs_parent_id`	`18279313228410085241`
`llmobs_trace_id`	`6af839c400000000a027947968c2eb53`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`process_id`	`3656`
`runtime-id`	`f441501157654b638091afc07b8ec261`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-helper · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Cost: $0.00116 on the wire, $1.75e-06 from the spans (-99.8%), at list price.
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-helper
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`0`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`7`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`0`
`gen_ai.usage.total_tokens`	`7`
`llmobs_parent_id`	`18279313228410085241`
`llmobs_trace_id`	`6af839c400000000a027947968c2eb53`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`process_id`	`3656`
`runtime-id`	`f441501157654b638091afc07b8ec261`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-helper · input_tokensVanishing tokens #
expected 2137, reported 7

Expected: 2137
Reported: 7
Read from: gen_ai.usage.input_tokens
Cost: $0.00116 on the wire, $1.75e-06 from the spans (-99.8%), at list price.
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-helper
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`0`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`7`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`0`
`gen_ai.usage.total_tokens`	`7`
`llmobs_parent_id`	`18279313228410085241`
`llmobs_trace_id`	`6af839c400000000a027947968c2eb53`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`process_id`	`3656`
`runtime-id`	`f441501157654b638091afc07b8ec261`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-helper · output_tokensVanishing tokens #
expected 313, reported 0

Expected: 313
Reported: 0
Read from: gen_ai.usage.output_tokens
Cost: $0.00116 on the wire, $1.75e-06 from the spans (-99.8%), at list price.
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-helper
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`0`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`7`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`0`
`gen_ai.usage.total_tokens`	`7`
`llmobs_parent_id`	`18279313228410085241`
`llmobs_trace_id`	`6af839c400000000a027947968c2eb53`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`process_id`	`3656`
`runtime-id`	`f441501157654b638091afc07b8ec261`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-no-usage · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-no-usage
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`7`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`14`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`7`
`gen_ai.usage.total_tokens`	`14`
`llmobs_parent_id`	`13835991631146762838`
`llmobs_trace_id`	`6af8398d00000000614d9d69842f9cec`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3029`
`runtime-id`	`c0e581e778eb43b2aea79b205b40f28a`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-no-usage · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-no-usage
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`7`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`14`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`7`
`gen_ai.usage.total_tokens`	`14`
`llmobs_parent_id`	`13835991631146762838`
`llmobs_trace_id`	`6af8398d00000000614d9d69842f9cec`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3029`
`runtime-id`	`c0e581e778eb43b2aea79b205b40f28a`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-no-usage · response_id #
expected chatcmpl-wt-nousage, reported nothing

Expected: chatcmpl-wt-nousage
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-no-usage
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`7`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`14`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`7`
`gen_ai.usage.total_tokens`	`14`
`llmobs_parent_id`	`13835991631146762838`
`llmobs_trace_id`	`6af8398d00000000614d9d69842f9cec`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3029`
`runtime-id`	`c0e581e778eb43b2aea79b205b40f28a`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-no-usage · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-no-usage
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`7`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`14`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`7`
`gen_ai.usage.total_tokens`	`14`
`llmobs_parent_id`	`13835991631146762838`
`llmobs_trace_id`	`6af8398d00000000614d9d69842f9cec`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3029`
`runtime-id`	`c0e581e778eb43b2aea79b205b40f28a`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-no-usage · input_tokensPhantom tokens #
expected nothing, reported 7: estimated: the stream carried no usage

Reported: 7
Read from: gen_ai.usage.input_tokens
Note: estimated: the stream carried no usage
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-no-usage
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`7`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`14`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`7`
`gen_ai.usage.total_tokens`	`14`
`llmobs_parent_id`	`13835991631146762838`
`llmobs_trace_id`	`6af8398d00000000614d9d69842f9cec`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3029`
`runtime-id`	`c0e581e778eb43b2aea79b205b40f28a`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-no-usage · output_tokensPhantom tokens #
expected nothing, reported 7: estimated: the stream carried no usage

Reported: 7
Read from: gen_ai.usage.output_tokens
Note: estimated: the stream carried no usage
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-no-usage
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`7`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`7`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`14`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7`
`gen_ai.usage.output_tokens`	`7`
`gen_ai.usage.total_tokens`	`14`
`llmobs_parent_id`	`13835991631146762838`
`llmobs_trace_id`	`6af8398d00000000614d9d69842f9cec`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3029`
`runtime-id`	`c0e581e778eb43b2aea79b205b40f28a`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-reasoning · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-reasoning
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2089`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2459`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`4548`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`gen_ai.usage.reasoning_output_tokens`	`1931`
`gen_ai.usage.total_tokens`	`4548`
`llmobs_parent_id`	`11427036633298410649`
`llmobs_trace_id`	`6af839bd000000006e1825373d5c8b0a`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3570`
`runtime-id`	`c1e5c3f9c6284ec0a2cc4da3a5048997`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-reasoning · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-reasoning
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2089`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2459`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`4548`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`gen_ai.usage.reasoning_output_tokens`	`1931`
`gen_ai.usage.total_tokens`	`4548`
`llmobs_parent_id`	`11427036633298410649`
`llmobs_trace_id`	`6af839bd000000006e1825373d5c8b0a`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3570`
`runtime-id`	`c1e5c3f9c6284ec0a2cc4da3a5048997`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-reasoning · response_id #
expected chatcmpl-wt-sreason, reported nothing

Expected: chatcmpl-wt-sreason
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-reasoning
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2089`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2459`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`4548`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`gen_ai.usage.reasoning_output_tokens`	`1931`
`gen_ai.usage.total_tokens`	`4548`
`llmobs_parent_id`	`11427036633298410649`
`llmobs_trace_id`	`6af839bd000000006e1825373d5c8b0a`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3570`
`runtime-id`	`c1e5c3f9c6284ec0a2cc4da3a5048997`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-reasoning · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-reasoning
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`2089`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`2459`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`4548`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`gen_ai.usage.reasoning_output_tokens`	`1931`
`gen_ai.usage.total_tokens`	`4548`
`llmobs_parent_id`	`11427036633298410649`
`llmobs_trace_id`	`6af839bd000000006e1825373d5c8b0a`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3570`
`runtime-id`	`c1e5c3f9c6284ec0a2cc4da3a5048997`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/stream-tool-call · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-tool-call
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1553`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`173`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1726`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1553`
`gen_ai.usage.output_tokens`	`173`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1726`
`llmobs_parent_id`	`9478784046724337125`
`llmobs_trace_id`	`6af8399b00000000f9b9c2187bbe2eb0`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3153`
`runtime-id`	`4406656b357a482bb473397e54658a86`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-tool-call · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-tool-call
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1553`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`173`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1726`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1553`
`gen_ai.usage.output_tokens`	`173`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1726`
`llmobs_parent_id`	`9478784046724337125`
`llmobs_trace_id`	`6af8399b00000000f9b9c2187bbe2eb0`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3153`
`runtime-id`	`4406656b357a482bb473397e54658a86`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-tool-call · response_id #
expected chatcmpl-wt-stools, reported nothing

Expected: chatcmpl-wt-stools
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-tool-call
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1553`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`173`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1726`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1553`
`gen_ai.usage.output_tokens`	`173`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1726`
`llmobs_parent_id`	`9478784046724337125`
`llmobs_trace_id`	`6af8399b00000000f9b9c2187bbe2eb0`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3153`
`runtime-id`	`4406656b357a482bb473397e54658a86`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/stream-tool-call · finish_reasonsSilent stop #
expected tool_calls, reported nothing

Expected: ['tool_calls']
Reproduce: spantruth run --subject ddtrace --scenario openai/stream-tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`1553`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`173`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1726`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1553`
`gen_ai.usage.output_tokens`	`173`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1726`
`llmobs_parent_id`	`9478784046724337125`
`llmobs_trace_id`	`6af8399b00000000f9b9c2187bbe2eb0`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3153`
`runtime-id`	`4406656b357a482bb473397e54658a86`

spantruth.scenario spantruth.driver, status UNSET

wrong ddtrace · openai/tool-call · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject ddtrace --scenario openai/tool-call
Rule: The model name the request asked for.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`931`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`88`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1019`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1019`
`llmobs_parent_id`	`16655175512711114227`
`llmobs_trace_id`	`6af8399400000000ea2f4aa1fd36de25`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3083`
`runtime-id`	`564c5e551734413cb56915df0fe19e6f`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/tool-call · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Reproduce: spantruth run --subject ddtrace --scenario openai/tool-call
Rule: The name of the model that generated the response.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`931`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`88`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1019`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1019`
`llmobs_parent_id`	`16655175512711114227`
`llmobs_trace_id`	`6af8399400000000ea2f4aa1fd36de25`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3083`
`runtime-id`	`564c5e551734413cb56915df0fe19e6f`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/tool-call · response_id #
expected chatcmpl-wt-tools, reported nothing

Expected: chatcmpl-wt-tools
Reproduce: spantruth run --subject ddtrace --scenario openai/tool-call
Rule: The unique identifier of the completion.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`931`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`88`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1019`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1019`
`llmobs_parent_id`	`16655175512711114227`
`llmobs_trace_id`	`6af8399400000000ea2f4aa1fd36de25`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3083`
`runtime-id`	`564c5e551734413cb56915df0fe19e6f`

spantruth.scenario spantruth.driver, status UNSET

missing ddtrace · openai/tool-call · finish_reasonsSilent stop #
expected tool_calls, reported nothing

Expected: ['tool_calls']
Reproduce: spantruth run --subject ddtrace --scenario openai/tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.request ddtrace.adapter, status UNSET

`_dd.llmobs.artificial_gen_ai_tags`	`true`
`_dd.llmobs.cache_read_input_tokens`	`0`
`_dd.llmobs.enabled`	`1`
`_dd.llmobs.input_tokens`	`931`
`_dd.llmobs.model_name`	`gpt-5-mini-2026-08-07`
`_dd.llmobs.model_provider`	`openai`
`_dd.llmobs.output_tokens`	`88`
`_dd.llmobs.span_kind`	`llm`
`_dd.llmobs.total_tokens`	`1019`
`_dd.measured`	`1`
`_dd.svc_src`	`m`
`component`	`openai`
`gen_ai.application.name`	`spantruth`
`gen_ai.operation.name`	`llm`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read_input_tokens`	`0`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`gen_ai.usage.reasoning_output_tokens`	`0`
`gen_ai.usage.total_tokens`	`1019`
`llmobs_parent_id`	`16655175512711114227`
`llmobs_trace_id`	`6af8399400000000ea2f4aa1fd36de25`
`openai.request.endpoint`	`/v1/chat/completions`
`openai.request.method`	`POST`
`openai.request.model`	`gpt-5-mini`
`openai.request.provider`	`OpenAI`
`openai.response.model`	`gpt-5-mini-2026-08-07`
`process_id`	`3083`
`runtime-id`	`564c5e551734413cb56915df0fe19e6f`

spantruth.scenario spantruth.driver, status UNSET

wrong langfuse · openai/async-stream-dropped · error.statusFalse OK #
expected ERROR, reported UNSET

Expected: ERROR
Reported: ['UNSET']
Upstream: reported traced_iterator finalizes in finally without level ERROR, so the span status stays UNSET. Fix proposed in https://github.com/langfuse/langfuse-python/pull/2056, which turns all three cells green.
Reproduce: spantruth run --subject langfuse --scenario openai/async-stream-dropped
Rule: Span status SHOULD follow Recording Errors: an operation that ended in an error sets the status to Error.

Exported spans (2)

OpenAI-generation langfuse-sdk, status UNSET

`langfuse.internal.is_app_root`	`True`
`langfuse.observation.completion_start_time`	`"2026-11-14T10:36:56.393886Z"`
`langfuse.observation.input`	`[{"role": "user", "content": "What is the capital of France?"}]`
`langfuse.observation.model.name`	`gpt-5-mini-2026-08-07`
`langfuse.observation.model.parameters`	`{"temperature": 1, "max_tokens": "Infinity", "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0, "service_tier": "default"}`
`langfuse.observation.output`	`The capital of France`
`langfuse.observation.type`	`generation`

spantruth.scenario spantruth.driver, status UNSET

missing langfuse · openai/async-stream-helper · span.recordedGhost span #
no LLM span arrived

Note: no LLM span arrived
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject langfuse --scenario openai/async-stream-helper
Rule: spantruth rule: a call the application made produces at least one span.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

wrong langfuse · openai/responses-stream-missing-finish-abort · error.statusFalse OK #
expected ERROR, reported UNSET

Expected: ERROR
Reported: ['UNSET']
Upstream: reported traced_iterator finalizes in finally without level ERROR, so the span status stays UNSET. Fix proposed in https://github.com/langfuse/langfuse-python/pull/2056, which turns all three cells green.
Reproduce: spantruth run --subject langfuse --scenario openai/responses-stream-missing-finish-abort
Rule: Span status SHOULD follow Recording Errors: an operation that ended in an error sets the status to Error.

Exported spans (2)

OpenAI-generation langfuse-sdk, status UNSET

`langfuse.internal.is_app_root`	`True`
`langfuse.observation.completion_start_time`	`"2026-11-14T10:37:00.939617Z"`
`langfuse.observation.input`	`[{"role": "user", "content": "What is the capital of France?"}]`
`langfuse.observation.metadata.background`	`false`
`langfuse.observation.metadata.id`	`"resp_025c23fff6653590006acbfc0086cc87d18124e70e8d3ff838"`
`langfuse.observation.metadata.max_output_tokens`	`16`
`langfuse.observation.metadata.metadata`	`{}`
`langfuse.observation.metadata.object`	`"response"`
`langfuse.observation.metadata.parallel_tool_calls`	`true`
`langfuse.observation.metadata.prompt_cache_retention`	`"24h"`
`langfuse.observation.metadata.reasoning`	`{"context":"all_turns","effort":"none","generate_summary":null,"mode":"standard","summary":null}`
`langfuse.observation.metadata.service_tier`	`"auto"`
`langfuse.observation.metadata.status`	`"in_progress"`
`langfuse.observation.metadata.temperature`	`1.0`
`langfuse.observation.metadata.tool_choice`	`"auto"`
`langfuse.observation.metadata.tool_usage`	`{"image_gen":{"input_tokens":0,"input_tokens_details":{"image_tokens":0,"text_tokens":0},"output_tokens":0,"total_tokens":0,"cached_tokens":null,"cached_tokens_details":null,"output_tokens_details":{"image_tokens":0,"text_tokens":0}},"web_search":{"num_requests":0}}`
`langfuse.observation.metadata.tools`	`[]`
`langfuse.observation.metadata.top_logprobs`	`0`
`langfuse.observation.metadata.top_p`	`0.98`
`langfuse.observation.metadata.truncation`	`"disabled"`
`langfuse.observation.model.name`	`gpt-6-luna`
`langfuse.observation.model.parameters`	`{"temperature": 1, "max_tokens": "Infinity", "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0, "service_tier": "auto"}`
`langfuse.observation.type`	`generation`

spantruth.scenario spantruth.driver, status UNSET

wrong langfuse · openai/stream-dropped · error.statusFalse OK #
expected ERROR, reported UNSET

Expected: ERROR
Reported: ['UNSET']
Upstream: reported traced_iterator finalizes in finally without level ERROR, so the span status stays UNSET. Fix proposed in https://github.com/langfuse/langfuse-python/pull/2056, which turns all three cells green.
Reproduce: spantruth run --subject langfuse --scenario openai/stream-dropped
Rule: Span status SHOULD follow Recording Errors: an operation that ended in an error sets the status to Error.

Exported spans (2)

OpenAI-generation langfuse-sdk, status UNSET

`langfuse.internal.is_app_root`	`True`
`langfuse.observation.completion_start_time`	`"2026-11-14T10:36:39.373902Z"`
`langfuse.observation.input`	`[{"role": "user", "content": "What is the capital of France?"}]`
`langfuse.observation.model.name`	`gpt-5-mini-2026-08-07`
`langfuse.observation.model.parameters`	`{"temperature": 1, "max_tokens": "Infinity", "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0, "service_tier": "default"}`
`langfuse.observation.output`	`The capital of France`
`langfuse.observation.type`	`generation`

spantruth.scenario spantruth.driver, status UNSET

missing langfuse · openai/stream-helper · span.recordedGhost span #
no LLM span arrived

Note: no LLM span arrived
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject langfuse --scenario openai/stream-helper
Rule: spantruth rule: a call the application made produces at least one span.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-basic · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 1187, "output_tokens": 263, "total_tokens": 1450, "cache_read_input_tokens": 0, "cache_creation_input_tokens": 0}`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5"}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "msg_wt_basic", "container": null, "content": [{"citations": null, "text": "The capital of France is Paris.", "type": "text"}], "diagnostics": null, "model": "claude-sonnet-5-5-20260915", "role": "assistant", "stop_details": null, "stop_reason": "end_turn", "stop_sequence": null, "type": "message", "usage": {"cache_creation": null, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "inference_geo": null, "input_tokens": 1187, "output_tokens": 263, "output_tokens_details": null, "server_tool_use": null, "service_tier": null, "speed": null}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-2fa23a0dd2f86d76d390b553c57c87e0"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-2fa23a0dd2f86d76d390b553c57c87e0"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}]}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_basic`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream
Rule: The name of the model that generated the response.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x7cf50fd82300>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream · response_id #
expected msg_wt_stream, reported nothing

Expected: msg_wt_stream
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream
Rule: The unique identifier of the completion.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x7cf50fd82300>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x7cf50fd82300>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream · input_tokensVanishing tokens #
expected 2311, reported nothing

Expected: 2311
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x7cf50fd82300>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream · output_tokensVanishing tokens #
expected 419, reported nothing

Expected: 419
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x7cf50fd82300>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1241accfa3aac3d579f36b0ba5ef4493"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · anthropic/async-stream-dropped · error.statusFalse OK #
expected ERROR, reported OK

Expected: ERROR
Reported: ['OK']
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream-dropped
Rule: Span status SHOULD follow Recording Errors: an operation that ended in an error sets the status to Error.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x79d6aca25b20>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-35088ce11cd1abab8c8cfdd01673c2e7"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-35088ce11cd1abab8c8cfdd01673c2e7"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream-dropped · error.typeFalse OK #

Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream-dropped
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x79d6aca25b20>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-35088ce11cd1abab8c8cfdd01673c2e7"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-35088ce11cd1abab8c8cfdd01673c2e7"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream-early-exit · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream-early-exit
Rule: The name of the model that generated the response.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x751d28e88200>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-651d9e2bb74c5da7002bb7e9fdd5bf9b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-651d9e2bb74c5da7002bb7e9fdd5bf9b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream-early-exit · response_id #
expected msg_wt_early, reported nothing

Expected: msg_wt_early
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream-early-exit
Rule: The unique identifier of the completion.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x751d28e88200>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-651d9e2bb74c5da7002bb7e9fdd5bf9b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-651d9e2bb74c5da7002bb7e9fdd5bf9b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream-early-exit · input_tokensVanishing tokens #
expected 1913, reported nothing

Expected: 1913
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream-early-exit
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x751d28e88200>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-651d9e2bb74c5da7002bb7e9fdd5bf9b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-651d9e2bb74c5da7002bb7e9fdd5bf9b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · anthropic/async-stream-error-event · error.statusFalse OK #
expected ERROR, reported OK

Expected: ERROR
Reported: ['OK']
Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream-error-event
Rule: Span status SHOULD follow Recording Errors: an operation that ended in an error sets the status to Error.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x7f82c844dac0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-e9fee79941e71ad55e518d88d6fb7bf5"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-e9fee79941e71ad55e518d88d6fb7bf5"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/async-stream-error-event · error.typeFalse OK #

Reproduce: spantruth run --subject mlflow --scenario anthropic/async-stream-error-event
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (4)

AsyncMessages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.AsyncStream object at 0x7f82c844dac0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-e9fee79941e71ad55e518d88d6fb7bf5"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-e9fee79941e71ad55e518d88d6fb7bf5"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/basic · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject mlflow --scenario anthropic/basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 1187, "output_tokens": 263, "total_tokens": 1450, "cache_read_input_tokens": 0, "cache_creation_input_tokens": 0}`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5"}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "msg_wt_basic", "container": null, "content": [{"citations": null, "text": "The capital of France is Paris.", "type": "text"}], "diagnostics": null, "model": "claude-sonnet-5-5-20260915", "role": "assistant", "stop_details": null, "stop_reason": "end_turn", "stop_sequence": null, "type": "message", "usage": {"cache_creation": null, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "inference_geo": null, "input_tokens": 1187, "output_tokens": 263, "output_tokens_details": null, "server_tool_use": null, "service_tier": null, "speed": null}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-d82a908c269b1fe72fe1529850e79ad6"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-d82a908c269b1fe72fe1529850e79ad6"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}]}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_basic`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/cache-hit · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Cost: $0.00785 on the wire, $0.0154 from the spans (+95.8%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 5802, "output_tokens": 377, "total_tokens": 6179, "cache_read_input_tokens": 4096, "cache_creation_input_tokens": 519}`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5"}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "msg_wt_cache", "container": null, "content": [{"citations": null, "text": "The capital of France is Paris.", "type": "text"}], "diagnostics": null, "model": "claude-sonnet-5-5-20260915", "role": "assistant", "stop_details": null, "stop_reason": "end_turn", "stop_sequence": null, "type": "message", "usage": {"cache_creation": null, "cache_creation_input_tokens": 519, "cache_read_input_tokens": 4096, "inference_geo": null, "input_tokens": 1187, "output_tokens": 377, "output_tokens_details": null, "server_tool_use": null, "service_tier": null, "speed": null}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-ecbce26d0f87be62eed4e058c3d98dc1"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-ecbce26d0f87be62eed4e058c3d98dc1"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}]}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_cache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`5802`
`gen_ai.usage.output_tokens`	`377`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/cache-hit · cache_read_tokensCache-blind tokens #
expected 4096, reported nothing

Expected: 4096
Cost: $0.00785 on the wire, $0.0154 from the spans (+95.8%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/cache-hit
Rule: Input tokens read from the provider cache.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 5802, "output_tokens": 377, "total_tokens": 6179, "cache_read_input_tokens": 4096, "cache_creation_input_tokens": 519}`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5"}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "msg_wt_cache", "container": null, "content": [{"citations": null, "text": "The capital of France is Paris.", "type": "text"}], "diagnostics": null, "model": "claude-sonnet-5-5-20260915", "role": "assistant", "stop_details": null, "stop_reason": "end_turn", "stop_sequence": null, "type": "message", "usage": {"cache_creation": null, "cache_creation_input_tokens": 519, "cache_read_input_tokens": 4096, "inference_geo": null, "input_tokens": 1187, "output_tokens": 377, "output_tokens_details": null, "server_tool_use": null, "service_tier": null, "speed": null}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-ecbce26d0f87be62eed4e058c3d98dc1"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-ecbce26d0f87be62eed4e058c3d98dc1"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}]}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_cache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`5802`
`gen_ai.usage.output_tokens`	`377`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/cache-hit · cache_write_tokensCache-blind tokens #
expected 519, reported nothing

Expected: 519
Cost: $0.00785 on the wire, $0.0154 from the spans (+95.8%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/cache-hit
Rule: Input tokens written to the provider cache.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 5802, "output_tokens": 377, "total_tokens": 6179, "cache_read_input_tokens": 4096, "cache_creation_input_tokens": 519}`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5"}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "msg_wt_cache", "container": null, "content": [{"citations": null, "text": "The capital of France is Paris.", "type": "text"}], "diagnostics": null, "model": "claude-sonnet-5-5-20260915", "role": "assistant", "stop_details": null, "stop_reason": "end_turn", "stop_sequence": null, "type": "message", "usage": {"cache_creation": null, "cache_creation_input_tokens": 519, "cache_read_input_tokens": 4096, "inference_geo": null, "input_tokens": 1187, "output_tokens": 377, "output_tokens_details": null, "server_tool_use": null, "service_tier": null, "speed": null}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-ecbce26d0f87be62eed4e058c3d98dc1"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-ecbce26d0f87be62eed4e058c3d98dc1"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}]}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_cache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`5802`
`gen_ai.usage.output_tokens`	`377`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/rate-limited · error.typeFalse OK #

Reproduce: spantruth run --subject mlflow --scenario anthropic/rate-limited
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (4)

Messages.create mlflow.tracing.provider, status ERROR

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5"}`
`mlflow.spanLogLevel`	`40`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-3bdae8a99390051685c8a5514d335f65"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-3bdae8a99390051685c8a5514d335f65"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status ERROR

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/retry-429 · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject mlflow --scenario anthropic/retry-429
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 2477, "output_tokens": 367, "total_tokens": 2844, "cache_read_input_tokens": 0, "cache_creation_input_tokens": 0}`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5"}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "msg_wt_retry", "container": null, "content": [{"citations": null, "text": "The capital of France is Paris.", "type": "text"}], "diagnostics": null, "model": "claude-sonnet-5-5-20260915", "role": "assistant", "stop_details": null, "stop_reason": "end_turn", "stop_sequence": null, "type": "message", "usage": {"cache_creation": null, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "inference_geo": null, "input_tokens": 2477, "output_tokens": 367, "output_tokens_details": null, "server_tool_use": null, "service_tier": null, "speed": null}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-72ac2b4d03adb2ee56af2099d9dc7d58"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-72ac2b4d03adb2ee56af2099d9dc7d58"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}]}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_retry`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream
Rule: The name of the model that generated the response.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73c50a52dbe0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream · response_id #
expected msg_wt_stream, reported nothing

Expected: msg_wt_stream
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream
Rule: The unique identifier of the completion.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73c50a52dbe0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73c50a52dbe0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream · input_tokensVanishing tokens #
expected 2311, reported nothing

Expected: 2311
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73c50a52dbe0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream · output_tokensVanishing tokens #
expected 419, reported nothing

Expected: 419
Cost: $0.00881 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73c50a52dbe0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-940cb72324354a6daa388c98c0957eb7"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-cumulative · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Cost: $0.00961 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-cumulative
Rule: The name of the model that generated the response.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73a74ed84d40>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-cumulative · response_id #
expected msg_wt_cumul, reported nothing

Expected: msg_wt_cumul
Cost: $0.00961 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-cumulative
Rule: The unique identifier of the completion.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73a74ed84d40>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-cumulative · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Cost: $0.00961 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-cumulative
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73a74ed84d40>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-cumulative · input_tokensVanishing tokens #
expected 5673, reported nothing

Expected: 5673
Cost: $0.00961 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-cumulative
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73a74ed84d40>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-cumulative · output_tokensVanishing tokens #
expected 467, reported nothing

Expected: 467
Cost: $0.00961 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-cumulative
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73a74ed84d40>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-cumulative · cache_read_tokensCache-blind tokens #
expected 3583, reported nothing

Expected: 3583
Cost: $0.00961 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-cumulative
Rule: Input tokens read from the provider cache.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73a74ed84d40>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-cumulative · cache_write_tokensCache-blind tokens #
expected 811, reported nothing

Expected: 811
Cost: $0.00961 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-cumulative
Rule: Input tokens written to the provider cache.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x73a74ed84d40>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-1b60ccc604ab3d8992c289dd7394324a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-hit · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Cost: $0.0107 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-hit
Rule: The name of the model that generated the response.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7e621b57b830>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-hit · response_id #
expected msg_wt_scache, reported nothing

Expected: msg_wt_scache
Cost: $0.0107 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-hit
Rule: The unique identifier of the completion.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7e621b57b830>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-hit · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Cost: $0.0107 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7e621b57b830>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-hit · input_tokensVanishing tokens #
expected 8424, reported nothing

Expected: 8424
Cost: $0.0107 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-hit
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7e621b57b830>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-hit · output_tokensVanishing tokens #
expected 511, reported nothing

Expected: 511
Cost: $0.0107 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-hit
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7e621b57b830>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-hit · cache_read_tokensCache-blind tokens #
expected 6144, reported nothing

Expected: 6144
Cost: $0.0107 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-hit
Rule: Input tokens read from the provider cache.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7e621b57b830>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-cache-hit · cache_write_tokensCache-blind tokens #
expected 737, reported nothing

Expected: 737
Cost: $0.0107 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-cache-hit
Rule: Input tokens written to the provider cache.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7e621b57b830>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-c1af742a915dfec9fed295cf2bca276b"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · anthropic/stream-dropped · error.statusFalse OK #
expected ERROR, reported OK

Expected: ERROR
Reported: ['OK']
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-dropped
Rule: Span status SHOULD follow Recording Errors: an operation that ended in an error sets the status to Error.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x750ce1575c70>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-8e0dcef29634dd83ce628ce2c1c2c224"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-8e0dcef29634dd83ce628ce2c1c2c224"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-dropped · error.typeFalse OK #

Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-dropped
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x750ce1575c70>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-8e0dcef29634dd83ce628ce2c1c2c224"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-8e0dcef29634dd83ce628ce2c1c2c224"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-early-exit · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-early-exit
Rule: The name of the model that generated the response.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7657a13116a0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-ae5df8fdf0aa0e4d8f938d0cdd367d7a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-ae5df8fdf0aa0e4d8f938d0cdd367d7a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-early-exit · response_id #
expected msg_wt_early, reported nothing

Expected: msg_wt_early
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-early-exit
Rule: The unique identifier of the completion.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7657a13116a0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-ae5df8fdf0aa0e4d8f938d0cdd367d7a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-ae5df8fdf0aa0e4d8f938d0cdd367d7a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-early-exit · input_tokensVanishing tokens #
expected 1913, reported nothing

Expected: 1913
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-early-exit
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7657a13116a0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-ae5df8fdf0aa0e4d8f938d0cdd367d7a"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-ae5df8fdf0aa0e4d8f938d0cdd367d7a"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-early-exit-with · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-early-exit-with
Rule: The name of the model that generated the response.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x78008937c9b0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-21cae34008b2a79e722797b425648ce4"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-21cae34008b2a79e722797b425648ce4"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-early-exit-with · response_id #
expected msg_wt_early, reported nothing

Expected: msg_wt_early
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-early-exit-with
Rule: The unique identifier of the completion.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x78008937c9b0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-21cae34008b2a79e722797b425648ce4"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-21cae34008b2a79e722797b425648ce4"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-early-exit-with · input_tokensVanishing tokens #
expected 1913, reported nothing

Expected: 1913
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-early-exit-with
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x78008937c9b0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-21cae34008b2a79e722797b425648ce4"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-21cae34008b2a79e722797b425648ce4"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · anthropic/stream-error-event · error.statusFalse OK #
expected ERROR, reported OK

Expected: ERROR
Reported: ['OK']
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-error-event
Rule: Span status SHOULD follow Recording Errors: an operation that ended in an error sets the status to Error.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x71e9489b57f0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-8b151c6acd1c8556536ca7c3e2c38008"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-8b151c6acd1c8556536ca7c3e2c38008"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-error-event · error.typeFalse OK #

Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-error-event
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x71e9489b57f0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-8b151c6acd1c8556536ca7c3e2c38008"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-8b151c6acd1c8556536ca7c3e2c38008"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-thinking · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Cost: $0.0181 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-thinking
Rule: The name of the model that generated the response.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7eb01ad94fb0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-thinking · response_id #
expected msg_wt_sthink, reported nothing

Expected: msg_wt_sthink
Cost: $0.0181 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-thinking
Rule: The unique identifier of the completion.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7eb01ad94fb0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-thinking · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Cost: $0.0181 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-thinking
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7eb01ad94fb0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-thinking · input_tokensVanishing tokens #
expected 1831, reported nothing

Expected: 1831
Cost: $0.0181 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-thinking
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7eb01ad94fb0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-thinking · output_tokensVanishing tokens #
expected 1439, reported nothing

Expected: 1439
Cost: $0.0181 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-thinking
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7eb01ad94fb0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-thinking · reasoning_tokensHidden reasoning #
expected 1103, reported nothing

Expected: 1103
Cost: $0.0181 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7eb01ad94fb0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-7316e232a30a1f1f4246f1e0aef7fac9"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-tool-call · response_model #
expected claude-sonnet-5-5-20260915, reported nothing

Expected: claude-sonnet-5-5-20260915
Cost: $0.00521 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-tool-call
Rule: The name of the model that generated the response.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tools`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true, "tools": [{"name": "get_weather", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"name": "get_time", "input_schema": {"type": "object", "properties": {"zone": {"type": "string"}}}}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7b84e35e96a0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.tool.definitions`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-tool-call · response_id #
expected msg_wt_stools, reported nothing

Expected: msg_wt_stools
Cost: $0.00521 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-tool-call
Rule: The unique identifier of the completion.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tools`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true, "tools": [{"name": "get_weather", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"name": "get_time", "input_schema": {"type": "object", "properties": {"zone": {"type": "string"}}}}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7b84e35e96a0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.tool.definitions`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-tool-call · finish_reasonsSilent stop #
expected tool_use, reported nothing

Expected: ['tool_use']
Cost: $0.00521 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tools`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true, "tools": [{"name": "get_weather", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"name": "get_time", "input_schema": {"type": "object", "properties": {"zone": {"type": "string"}}}}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7b84e35e96a0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.tool.definitions`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-tool-call · input_tokensVanishing tokens #
expected 1619, reported nothing

Expected: 1619
Cost: $0.00521 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-tool-call
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tools`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true, "tools": [{"name": "get_weather", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"name": "get_time", "input_schema": {"type": "object", "properties": {"zone": {"type": "string"}}}}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7b84e35e96a0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.tool.definitions`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/stream-tool-call · output_tokensVanishing tokens #
expected 197, reported nothing

Expected: 197
Cost: $0.00521 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario anthropic/stream-tool-call
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tools`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "stream": true, "tools": [{"name": "get_weather", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"name": "get_time", "input_schema": {"type": "object", "properties": {"zone": {"type": "string"}}}}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<anthropic.Stream object at 0x7b84e35e96a0>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-b4485d2c9c542167d8101313073aae60"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.tool.definitions`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/thinking · finish_reasonsSilent stop #
expected end_turn, reported nothing

Expected: ['end_turn']
Reproduce: spantruth run --subject mlflow --scenario anthropic/thinking
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 1627, "output_tokens": 1273, "total_tokens": 2900, "cache_read_input_tokens": 0, "cache_creation_input_tokens": 0}`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5"}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "msg_wt_thinking", "container": null, "content": [{"signature": "dGVzdC1zaWduYXR1cmU=", "thinking": "Let me think about the capital of France. The capital of France is Paris.", "type": "thinking"}, {"citations": null, "text": "The capital of France is Paris.", "type": "text"}], "diagnostics": null, "model": "claude-sonnet-5-5-20260915", "role": "assistant", "stop_details": null, "stop_reason": "end_turn", "stop_sequence": null, "type": "message", "usage": {"cache_creation": null, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "inference_geo": null, "input_tokens": 1627, "output_tokens": 1273, "output_tokens_details": {"thinking_tokens": 947}, "server_tool_use": null, "service_tier": null, "speed": null}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-e2590f4595ae3426d1bec8437dde981d"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-e2590f4595ae3426d1bec8437dde981d"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "{\"signature\": \"dGVzdC1zaWduYXR1cmU=\", \"thinking\": \"Let me think about the capital of France. The capital of France is Paris.\", \"type\": \"thinking\"}"}, {"type": "text", "content": "The capital of France is Paris."}]}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_thinking`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/thinking · reasoning_tokensHidden reasoning #
expected 947, reported nothing

Expected: 947
Reproduce: spantruth run --subject mlflow --scenario anthropic/thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 1627, "output_tokens": 1273, "total_tokens": 2900, "cache_read_input_tokens": 0, "cache_creation_input_tokens": 0}`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5"}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "msg_wt_thinking", "container": null, "content": [{"signature": "dGVzdC1zaWduYXR1cmU=", "thinking": "Let me think about the capital of France. The capital of France is Paris.", "type": "thinking"}, {"citations": null, "text": "The capital of France is Paris.", "type": "text"}], "diagnostics": null, "model": "claude-sonnet-5-5-20260915", "role": "assistant", "stop_details": null, "stop_reason": "end_turn", "stop_sequence": null, "type": "message", "usage": {"cache_creation": null, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "inference_geo": null, "input_tokens": 1627, "output_tokens": 1273, "output_tokens_details": {"thinking_tokens": 947}, "server_tool_use": null, "service_tier": null, "speed": null}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-e2590f4595ae3426d1bec8437dde981d"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-e2590f4595ae3426d1bec8437dde981d"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "{\"signature\": \"dGVzdC1zaWduYXR1cmU=\", \"thinking\": \"Let me think about the capital of France. The capital of France is Paris.\", \"type\": \"thinking\"}"}, {"type": "text", "content": "The capital of France is Paris."}]}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_thinking`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · anthropic/tool-call · finish_reasonsSilent stop #
expected tool_use, reported nothing

Expected: ['tool_use']
Reproduce: spantruth run --subject mlflow --scenario anthropic/tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Messages.create mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 931, "output_tokens": 88, "total_tokens": 1019, "cache_read_input_tokens": 0, "cache_creation_input_tokens": 0}`
`mlflow.chat.tools`	`[{"type": "function", "function": {"name": "get_weather", "description": null, "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "description": null, "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`
`mlflow.llm.model`	`"claude-sonnet-5-5"`
`mlflow.llm.provider`	`"anthropic"`
`mlflow.message.format`	`"anthropic"`
`mlflow.spanInputs`	`{"max_tokens": 1024, "messages": [{"role": "user", "content": "What is the capital of France?"}], "model": "claude-sonnet-5-5", "tools": [{"name": "get_weather", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"name": "get_time", "input_schema": {"type": "object", "properties": {"zone": {"type": "string"}}}}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "msg_wt_tools", "container": null, "content": [{"id": "toolu_wt_1", "caller": null, "input": {"city": "Paris"}, "name": "get_weather", "type": "tool_use", "toolset_name": null}, {"id": "toolu_wt_2", "caller": null, "input": {"zone": "Europe/Paris"}, "name": "get_time", "type": "tool_use", "toolset_name": null}], "diagnostics": null, "model": "claude-sonnet-5-5-20260915", "role": "assistant", "stop_details": null, "stop_reason": "tool_use", "stop_sequence": null, "type": "message", "usage": {"cache_creation": null, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0, "inference_geo": null, "input_tokens": 931, "output_tokens": 88, "output_tokens_details": null, "server_tool_use": null, "service_tier": null, "speed": null}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-f79fb82983d0fb6099444802a25fe084"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-f79fb82983d0fb6099444802a25fe084"`

chat claude-sonnet-5-5 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "tool_call", "id": "toolu_wt_1", "name": "get_weather", "arguments": {"city": "Paris"}}, {"type": "tool_call", "id": "toolu_wt_2", "name": "get_time", "arguments": {"zone": "Europe/Paris"}}]}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_tools`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.tool.definitions`	`[{"name": "get_weather", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"name": "get_time", "input_schema": {"type": "object", "properties": {"zone": {"type": "string"}}}}]`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/async-basic · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/async-basic
Rule: The model name the request asked for.

Exported spans (4)

AsyncCompletions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 1187, "output_tokens": 263, "total_tokens": 1450, "cache_read_input_tokens": 0}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-basic", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": [], "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": "default", "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 263, "prompt_tokens": 1187, "total_tokens": 1450, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-73fcbd88fa8f7a53fe6ce2a885e00665"`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-73fcbd88fa8f7a53fe6ce2a885e00665"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-basic`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/async-stream · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/async-stream
Rule: The model name the request asked for.

Exported spans (4)

AsyncCompletions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 2311, "output_tokens": 419, "total_tokens": 2730, "cache_read_input_tokens": 0}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-stream", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 419, "prompt_tokens": 2311, "total_tokens": 2730, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-a33f5527acab603d7abc2fa300b63784"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-a33f5527acab603d7abc2fa300b63784"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-stream`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/async-stream-dropped · error.typeFalse OK #

Reproduce: spantruth run --subject mlflow --scenario openai/async-stream-dropped
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (4)

AsyncCompletions mlflow.tracing.provider, status ERROR

`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`40`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-6856fb710e2d7b4004726f73a3e87885"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-6856fb710e2d7b4004726f73a3e87885"`

chat mlflow.tracing.provider, status ERROR

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/async-stream-early-exit · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/async-stream-early-exit
Rule: The model name the request asked for.

Exported spans (4)

AsyncCompletions mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`{"id": "chatcmpl-wt-early", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": null}`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-f96b01597b568587dd2a44600c07c73e"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-f96b01597b568587dd2a44600c07c73e"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital"}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-early`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/async-stream-helper · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/async-stream-helper
Rule: The name of the model that generated the response.

Exported spans (4)

AsyncCompletions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<AsyncAPIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/async-stream-helper · response_id #
expected chatcmpl-wt-shelper, reported nothing

Expected: chatcmpl-wt-shelper
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/async-stream-helper
Rule: The unique identifier of the completion.

Exported spans (4)

AsyncCompletions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<AsyncAPIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/async-stream-helper · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/async-stream-helper
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

AsyncCompletions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<AsyncAPIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/async-stream-helper · input_tokensVanishing tokens #
expected 2137, reported nothing

Expected: 2137
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/async-stream-helper
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

AsyncCompletions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<AsyncAPIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/async-stream-helper · output_tokensVanishing tokens #
expected 313, reported nothing

Expected: 313
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/async-stream-helper
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (4)

AsyncCompletions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<AsyncAPIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-bd1ce321085447d4ad2b942ccbca48f8"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/basic · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/basic
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 1187, "output_tokens": 263, "total_tokens": 1450, "cache_read_input_tokens": 0}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-basic", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": [], "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": "default", "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 263, "prompt_tokens": 1187, "total_tokens": 1450, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-9a0b8bee52a3f567fa7164c02c124e5c"`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-9a0b8bee52a3f567fa7164c02c124e5c"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-basic`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/cache-hit · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Cost: $0.00118 on the wire, $0.0021 from the spans (+78.3%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/cache-hit
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 5379, "output_tokens": 377, "total_tokens": 5756, "cache_read_input_tokens": 4096}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-cache", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": [], "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": "default", "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 377, "prompt_tokens": 5379, "total_tokens": 5756, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 4096, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-4144f3260264baa9fef7a656f647da17"`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-4144f3260264baa9fef7a656f647da17"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-cache`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`5379`
`gen_ai.usage.output_tokens`	`377`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/cache-hit · cache_read_tokensCache-blind tokens #
expected 4096, reported nothing

Expected: 4096
Cost: $0.00118 on the wire, $0.0021 from the spans (+78.3%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/cache-hit
Rule: Input tokens read from the provider cache.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 5379, "output_tokens": 377, "total_tokens": 5756, "cache_read_input_tokens": 4096}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-cache", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": [], "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": "default", "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 377, "prompt_tokens": 5379, "total_tokens": 5756, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 4096, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-4144f3260264baa9fef7a656f647da17"`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-4144f3260264baa9fef7a656f647da17"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-cache`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`5379`
`gen_ai.usage.output_tokens`	`377`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/rate-limited · error.typeFalse OK #

Reproduce: spantruth run --subject mlflow --scenario openai/rate-limited
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (4)

Completions mlflow.tracing.provider, status ERROR

`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}`
`mlflow.spanLogLevel`	`40`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-75f0781f428eb76694da48bca0ddd692"`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-75f0781f428eb76694da48bca0ddd692"`

chat mlflow.tracing.provider, status ERROR

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/reasoning · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/reasoning
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 1601, "output_tokens": 2219, "total_tokens": 3820, "cache_read_input_tokens": 0}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-reason", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": [], "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": "default", "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 2219, "prompt_tokens": 1601, "total_tokens": 3820, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 1792, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-f10fa538695bcbc39ce015c83066d78b"`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-f10fa538695bcbc39ce015c83066d78b"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-reason`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`1601`
`gen_ai.usage.output_tokens`	`2219`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/reasoning · reasoning_tokensHidden reasoning #
expected 1792, reported nothing

Expected: 1792
Reproduce: spantruth run --subject mlflow --scenario openai/reasoning
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 1601, "output_tokens": 2219, "total_tokens": 3820, "cache_read_input_tokens": 0}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-reason", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": [], "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": "default", "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 2219, "prompt_tokens": 1601, "total_tokens": 3820, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 1792, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-f10fa538695bcbc39ce015c83066d78b"`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-f10fa538695bcbc39ce015c83066d78b"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-reason`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`1601`
`gen_ai.usage.output_tokens`	`2219`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/responses-stream-missing-finish-abort · error.typeFalse OK #

Reproduce: spantruth run --subject mlflow --scenario openai/responses-stream-missing-finish-abort
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (4)

Responses mlflow.tracing.provider, status ERROR

`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-6-luna", "input": [{"role": "user", "content": "What is the capital of France?"}], "stream": true}`
`mlflow.spanLogLevel`	`40`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-000d4ac3935a4b6cc391011b4c49d01b"`
`model`	`"gpt-6-luna"`
`stream`	`true`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-000d4ac3935a4b6cc391011b4c49d01b"`

chat mlflow.tracing.provider, status ERROR

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`model`	`"gpt-6-luna"`
`stream`	`true`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/retry-429 · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/retry-429
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 2477, "output_tokens": 367, "total_tokens": 2844, "cache_read_input_tokens": 0}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-retry", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": [], "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": "default", "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 367, "prompt_tokens": 2477, "total_tokens": 2844, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-ded95af2c1a033d933f3d580585ac37f"`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-ded95af2c1a033d933f3d580585ac37f"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-retry`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`model`	`"gpt-5-mini"`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/stream · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/stream
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 2311, "output_tokens": 419, "total_tokens": 2730, "cache_read_input_tokens": 0}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-stream", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 419, "prompt_tokens": 2311, "total_tokens": 2730, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-03cb7dfdbb3d0852b81c8bb06c3d3c0d"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-03cb7dfdbb3d0852b81c8bb06c3d3c0d"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-stream`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/stream-cache-hit · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Cost: $0.00144 on the wire, $0.00283 from the spans (+95.8%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/stream-cache-hit
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 7213, "output_tokens": 511, "total_tokens": 7724, "cache_read_input_tokens": 6144}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-scache", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 511, "prompt_tokens": 7213, "total_tokens": 7724, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 6144, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-e1c10f51855a34355ed8138d5155fa0f"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-e1c10f51855a34355ed8138d5155fa0f"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-scache`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7213`
`gen_ai.usage.output_tokens`	`511`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/stream-cache-hit · cache_read_tokensCache-blind tokens #
expected 6144, reported nothing

Expected: 6144
Cost: $0.00144 on the wire, $0.00283 from the spans (+95.8%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/stream-cache-hit
Rule: Input tokens read from the provider cache.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 7213, "output_tokens": 511, "total_tokens": 7724, "cache_read_input_tokens": 6144}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-scache", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 511, "prompt_tokens": 7213, "total_tokens": 7724, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 6144, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-e1c10f51855a34355ed8138d5155fa0f"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-e1c10f51855a34355ed8138d5155fa0f"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-scache`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`7213`
`gen_ai.usage.output_tokens`	`511`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/stream-dropped · error.typeFalse OK #

Reproduce: spantruth run --subject mlflow --scenario openai/stream-dropped
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (4)

Completions mlflow.tracing.provider, status ERROR

`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`40`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-0f5a520d49f23f7662435d6ea8572437"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-0f5a520d49f23f7662435d6ea8572437"`

chat mlflow.tracing.provider, status ERROR

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/stream-early-exit · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/stream-early-exit
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`{"id": "chatcmpl-wt-early", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": null}`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-28474ebde9c71f5193a373e4952a983b"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-28474ebde9c71f5193a373e4952a983b"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital"}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-early`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/stream-early-exit-with · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/stream-early-exit-with
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`{"id": "chatcmpl-wt-early", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": null}`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-85db18879fc303084c3b72d9488a1a0b"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-85db18879fc303084c3b72d9488a1a0b"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital"}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-early`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/stream-helper · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/stream-helper
Rule: The name of the model that generated the response.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/stream-helper · response_id #
expected chatcmpl-wt-shelper, reported nothing

Expected: chatcmpl-wt-shelper
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/stream-helper
Rule: The unique identifier of the completion.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/stream-helper · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/stream-helper
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/stream-helper · input_tokensVanishing tokens #
expected 2137, reported nothing

Expected: 2137
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/stream-helper
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/stream-helper · output_tokensVanishing tokens #
expected 313, reported nothing

Expected: 313
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject mlflow --scenario openai/stream-helper
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`mlflow.llm.model`	`"gpt-5-mini"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}, "extra_headers": {"X-Stainless-Raw-Response": "stream"}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`"<APIResponse [200 OK] type=<class 'openai.types.chat.chat_completion.ChatCompletion'>>"`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-d5342063783b5f565fa040dda4c0fec7"`

chat gpt-5-mini mlflow.tracing.provider, status OK

`extra_headers`	`{"X-Stainless-Raw-Response": "stream"}`
`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/stream-no-usage · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/stream-no-usage
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	`{"id": "chatcmpl-wt-nousage", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": null}`
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-e5421077ed1f32ee436e9c3342146327"`
`model`	`"gpt-5-mini"`
`stream`	`true`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-e5421077ed1f32ee436e9c3342146327"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-nousage`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`model`	`"gpt-5-mini"`
`stream`	`true`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/stream-reasoning · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/stream-reasoning
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 2089, "output_tokens": 2459, "total_tokens": 4548, "cache_read_input_tokens": 0}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-sreason", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 2459, "prompt_tokens": 2089, "total_tokens": 4548, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 1931, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-05872a1d3888cd5bab623e916de222e2"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-05872a1d3888cd5bab623e916de222e2"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-sreason`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

missing mlflow · openai/stream-reasoning · reasoning_tokensHidden reasoning #
expected 1931, reported nothing

Expected: 1931
Reproduce: spantruth run --subject mlflow --scenario openai/stream-reasoning
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 2089, "output_tokens": 2459, "total_tokens": 4548, "cache_read_input_tokens": 0}`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-sreason", "choices": [{"finish_reason": "stop", "index": 0, "logprobs": null, "message": {"content": "The capital of France is Paris.", "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": null}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 2459, "prompt_tokens": 2089, "total_tokens": 4548, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 1931, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-05872a1d3888cd5bab623e916de222e2"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-05872a1d3888cd5bab623e916de222e2"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-sreason`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/stream-tool-call · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/stream-tool-call
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 1553, "output_tokens": 173, "total_tokens": 1726, "cache_read_input_tokens": 0}`
`mlflow.chat.tools`	`[{"type": "function", "function": {"name": "get_weather", "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}, {"type": "function", "function": {"name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}}], "stream": true, "stream_options": {"include_usage": true}}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-stools", "choices": [{"finish_reason": "tool_calls", "index": 0, "logprobs": null, "message": {"content": null, "refusal": null, "role": "assistant", "annotations": null, "audio": null, "function_call": null, "tool_calls": [{"id": "call_wt_1", "function": {"arguments": "{\"city\":\"Paris\"}", "name": "get_weather"}, "type": "function"}, {"id": "call_wt_2", "function": {"arguments": "{\"zone\":\"Europe/Paris\"}", "name": "get_time"}, "type": "function"}]}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": null, "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 173, "prompt_tokens": 1553, "total_tokens": 1726, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-5d0408bd662adb51437d06d3a5cba0ef"`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`
`tools`	`[{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}, {"type": "function", "function": {"name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}}]`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-5d0408bd662adb51437d06d3a5cba0ef"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "tool_call", "id": "call_wt_1", "name": "get_weather", "arguments": {"city": "Paris"}}, {"type": "tool_call", "id": "call_wt_2", "name": "get_time", "arguments": {"zone": "Europe/Paris"}}], "finish_reason": "tool_calls"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['tool_calls']`
`gen_ai.response.id`	`chatcmpl-wt-stools`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.tool.definitions`	`[{"type": "function", "name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"type": "function", "name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}]`
`gen_ai.usage.input_tokens`	`1553`
`gen_ai.usage.output_tokens`	`173`
`model`	`"gpt-5-mini"`
`stream`	`true`
`stream_options`	`{"include_usage": true}`
`tools`	`[{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}, {"type": "function", "function": {"name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}}]`

spantruth.scenario spantruth.driver, status UNSET

wrong mlflow · openai/tool-call · request_model #
expected gpt-5-mini, reported gpt-5-mini-2026-08-07

Expected: gpt-5-mini
Reported: gpt-5-mini-2026-08-07
Read from: gen_ai.request.model
Reproduce: spantruth run --subject mlflow --scenario openai/tool-call
Rule: The model name the request asked for.

Exported spans (4)

Completions mlflow.tracing.provider, status OK

`mlflow.chat.tokenUsage`	`{"input_tokens": 931, "output_tokens": 88, "total_tokens": 1019, "cache_read_input_tokens": 0}`
`mlflow.chat.tools`	`[{"type": "function", "function": {"name": "get_weather", "parameters": {"properties": {"city": {"type": "string"}}, "type": "object"}}}, {"type": "function", "function": {"name": "get_time", "parameters": {"properties": {"zone": {"type": "string"}}, "type": "object"}}}]`
`mlflow.llm.model`	`"gpt-5-mini-2026-08-07"`
`mlflow.message.format`	`"openai"`
`mlflow.spanInputs`	`{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}], "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}, {"type": "function", "function": {"name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}}]}`
`mlflow.spanLogLevel`	`20`
`mlflow.spanOutputs`	{"id": "chatcmpl-wt-tools", "choices": [{"finish_reason": "tool_calls", "index": 0, "logprobs": null, "message": {"content": null, "refusal": null, "role": "assistant", "annotations": [], "audio": null, "function_call": null, "tool_calls": [{"id": "call_wt_1", "function": {"arguments": "{\"city\":\"Paris\"}", "name": "get_weather"}, "type": "function"}, {"id": "call_wt_2", "function": {"arguments": "{\"zone\":\"Europe/Paris\"}", "name": "get_time"}, "type": "function"}]}}], "created": 1760000000, "model": "gpt-5-mini-2026-08-07", "object": "chat.completion", "metadata": null, "moderation": null, "service_tier": "default", "system_fingerprint": "fp_wt0001", "usage": {"completion_tokens": 88, "prompt_tokens": 931, "total_tokens": 1019, "completion_tokens_details": {"accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0, "text_tokens": null}, "prompt_tokens_details": {"audio_tokens": 0, "cache_write_tokens": null, "cached_tokens": 0, "image_tokens": null, "text_tokens": null}}}
`mlflow.spanType`	`"CHAT_MODEL"`
`mlflow.traceRequestId`	`"tr-15f8cdac4844971710f065ea49b026f6"`
`model`	`"gpt-5-mini"`
`tools`	`[{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}, {"type": "function", "function": {"name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}}]`

spantruth.scenario spantruth.driver, status UNSET

`mlflow.spanType`	`null`
`mlflow.traceRequestId`	`"tr-15f8cdac4844971710f065ea49b026f6"`

chat gpt-5-mini-2026-08-07 mlflow.tracing.provider, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "tool_call", "id": "call_wt_1", "name": "get_weather", "arguments": {"city": "Paris"}}, {"type": "tool_call", "id": "call_wt_2", "name": "get_time", "arguments": {"zone": "Europe/Paris"}}], "finish_reason": "tool_calls"}]`
`gen_ai.request.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.finish_reasons`	`['tool_calls']`
`gen_ai.response.id`	`chatcmpl-wt-tools`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.tool.definitions`	`[{"type": "function", "name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"type": "function", "name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}]`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`model`	`"gpt-5-mini"`
`tools`	`[{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}, {"type": "function", "function": {"name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}}]`

spantruth.scenario spantruth.driver, status UNSET

missing openinference · openai/async-stream-helper · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openinference --scenario openai/async-stream-helper
Rule: The name of the model that generated the response.

Exported spans (2)

ChatCompletion openinference.instrumentation.openai, status OK

`input.mime_type`	`application/json`
`input.value`	`{"model": "gpt-5-mini", "stream": true, "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream_options": {"include_usage": true}}`
`llm.input_messages.0.message.content`	`What is the capital of France?`
`llm.input_messages.0.message.role`	`user`
`llm.invocation_parameters`	`{"model": "gpt-5-mini", "stream": true, "stream_options": {"include_usage": true}}`
`llm.system`	`openai`
`openinference.span.kind`	`LLM`
`output.value`	`<coroutine object AsyncAPIResponse.parse at 0x738f515ed7b0>`

spantruth.scenario spantruth.driver, status UNSET

missing openinference · openai/async-stream-helper · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openinference --scenario openai/async-stream-helper
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

ChatCompletion openinference.instrumentation.openai, status OK

`input.mime_type`	`application/json`
`input.value`	`{"model": "gpt-5-mini", "stream": true, "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream_options": {"include_usage": true}}`
`llm.input_messages.0.message.content`	`What is the capital of France?`
`llm.input_messages.0.message.role`	`user`
`llm.invocation_parameters`	`{"model": "gpt-5-mini", "stream": true, "stream_options": {"include_usage": true}}`
`llm.system`	`openai`
`openinference.span.kind`	`LLM`
`output.value`	`<coroutine object AsyncAPIResponse.parse at 0x738f515ed7b0>`

spantruth.scenario spantruth.driver, status UNSET

missing openinference · openai/async-stream-helper · input_tokensVanishing tokens #
expected 2137, reported nothing

Expected: 2137
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openinference --scenario openai/async-stream-helper
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (2)

ChatCompletion openinference.instrumentation.openai, status OK

`input.mime_type`	`application/json`
`input.value`	`{"model": "gpt-5-mini", "stream": true, "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream_options": {"include_usage": true}}`
`llm.input_messages.0.message.content`	`What is the capital of France?`
`llm.input_messages.0.message.role`	`user`
`llm.invocation_parameters`	`{"model": "gpt-5-mini", "stream": true, "stream_options": {"include_usage": true}}`
`llm.system`	`openai`
`openinference.span.kind`	`LLM`
`output.value`	`<coroutine object AsyncAPIResponse.parse at 0x738f515ed7b0>`

spantruth.scenario spantruth.driver, status UNSET

missing openinference · openai/async-stream-helper · output_tokensVanishing tokens #
expected 313, reported nothing

Expected: 313
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openinference --scenario openai/async-stream-helper
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

ChatCompletion openinference.instrumentation.openai, status OK

`input.mime_type`	`application/json`
`input.value`	`{"model": "gpt-5-mini", "stream": true, "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream_options": {"include_usage": true}}`
`llm.input_messages.0.message.content`	`What is the capital of France?`
`llm.input_messages.0.message.role`	`user`
`llm.invocation_parameters`	`{"model": "gpt-5-mini", "stream": true, "stream_options": {"include_usage": true}}`
`llm.system`	`openai`
`openinference.span.kind`	`LLM`
`output.value`	`<coroutine object AsyncAPIResponse.parse at 0x738f515ed7b0>`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · anthropic/async-stream-dropped · span.endedGhost span #
never ended: chat claude-sonnet-5-5

Note: never ended: chat claude-sonnet-5-5
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/async-stream-dropped
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--async-stream-dropped/v1/messages`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · anthropic/async-stream-early-exit · span.endedGhost span #
never ended: chat claude-sonnet-5-5

Note: never ended: chat claude-sonnet-5-5
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/async-stream-early-exit
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--async-stream-early-exit/v1/messages`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · anthropic/async-stream-error-event · span.endedGhost span #
never ended: chat claude-sonnet-5-5

Note: never ended: chat claude-sonnet-5-5
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/async-stream-error-event
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--async-stream-error-event/v1/messages`

spantruth.scenario spantruth.driver, status UNSET

wrong openlit · anthropic/cache-hit · input_tokensCache-blind tokens #
expected 5802, reported 1187: cache reads and writes left out

Expected: 5802
Reported: 1187
Read from: gen_ai.usage.input_tokens
Note: cache reads and writes left out
Cost: $0.00785 on the wire, $0.00548 from the spans (-30.2%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/cache-hit
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--cache-hit/v1/messages`

chat claude-sonnet-5-5 openlit.instrumentation.anthropic, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`1564`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stop_sequences`	`[]`
`gen_ai.request.stream`	`False`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_k`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.response.finish_reasons`	`['end_turn']`
`gen_ai.response.id`	`msg_wt_cache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.sdk.version`	`0.125.0`
`gen_ai.server.time_per_output_token`	`0`
`gen_ai.server.time_to_first_token`	`0.013194561004638672`
`gen_ai.usage.cache_creation.input_tokens`	`519`
`gen_ai.usage.cache_read.input_tokens`	`4096`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`377`
`openlit.agent.version_hash`	`126a1732f5faa9e0`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

missing openlit · anthropic/rate-limited · span.recordedGhost span #
no LLM span arrived

Note: no LLM span arrived
Reproduce: spantruth run --subject openlit --scenario anthropic/rate-limited
Rule: spantruth rule: a call the application made produces at least one span.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status ERROR

`http.method`	`POST`
`http.status_code`	`429`
`http.url`	`http://127.0.0.1:46433/s/anthropic--rate-limited/v1/messages`

chat claude-sonnet-5-5 openlit.instrumentation.anthropic, status ERROR

spantruth.scenario spantruth.driver, status UNSET

wrong openlit · anthropic/stream-cache-cumulative · input_tokensCache-blind tokens #
expected 5673, reported 1279: cache reads and writes left out

Expected: 5673
Reported: 1279
Read from: gen_ai.usage.input_tokens
Note: cache reads and writes left out
Cost: $0.00961 on the wire, $0.00706 from the spans (-26.6%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/stream-cache-cumulative
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--stream-cache-cumulative/v1/messages`

chat claude-sonnet-5-5 openlit.instrumentation.anthropic, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`1746`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stop_sequences`	`[]`
`gen_ai.request.stream`	`True`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_k`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.response.finish_reasons`	`['end_turn']`
`gen_ai.response.id`	`msg_wt_cumul`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.sdk.version`	`0.125.0`
`gen_ai.server.time_per_output_token`	`0.0017023086547851562`
`gen_ai.server.time_to_first_token`	`0.008288145065307617`
`gen_ai.usage.cache_creation.input_tokens`	`811`
`gen_ai.usage.cache_read.input_tokens`	`3583`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`1279`
`gen_ai.usage.output_tokens`	`467`
`openlit.agent.version_hash`	`126a1732f5faa9e0`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

wrong openlit · anthropic/stream-cache-hit · input_tokensCache-blind tokens #
expected 8424, reported 1543: cache reads and writes left out

Expected: 8424
Reported: 1543
Read from: gen_ai.usage.input_tokens
Note: cache reads and writes left out
Cost: $0.0107 on the wire, $0.0082 from the spans (-23.1%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/stream-cache-hit
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--stream-cache-hit/v1/messages`

chat claude-sonnet-5-5 openlit.instrumentation.anthropic, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`2054`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stop_sequences`	`[]`
`gen_ai.request.stream`	`True`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_k`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.response.finish_reasons`	`['end_turn']`
`gen_ai.response.id`	`msg_wt_scache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.sdk.version`	`0.125.0`
`gen_ai.server.time_per_output_token`	`0.0010051727294921875`
`gen_ai.server.time_to_first_token`	`0.005663871765136719`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`1543`
`gen_ai.usage.output_tokens`	`511`
`openlit.agent.version_hash`	`126a1732f5faa9e0`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

wrong openlit · anthropic/stream-cache-hit · cache_read_tokensCache-blind tokens #
expected 6144, reported 0

Expected: 6144
Reported: 0
Read from: gen_ai.usage.cache_read.input_tokens
Cost: $0.0107 on the wire, $0.0082 from the spans (-23.1%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/stream-cache-hit
Rule: Input tokens read from the provider cache.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--stream-cache-hit/v1/messages`

chat claude-sonnet-5-5 openlit.instrumentation.anthropic, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`2054`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stop_sequences`	`[]`
`gen_ai.request.stream`	`True`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_k`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.response.finish_reasons`	`['end_turn']`
`gen_ai.response.id`	`msg_wt_scache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.sdk.version`	`0.125.0`
`gen_ai.server.time_per_output_token`	`0.0010051727294921875`
`gen_ai.server.time_to_first_token`	`0.005663871765136719`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`1543`
`gen_ai.usage.output_tokens`	`511`
`openlit.agent.version_hash`	`126a1732f5faa9e0`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

wrong openlit · anthropic/stream-cache-hit · cache_write_tokensCache-blind tokens #
expected 737, reported 0

Expected: 737
Reported: 0
Read from: gen_ai.usage.cache_creation.input_tokens
Cost: $0.0107 on the wire, $0.0082 from the spans (-23.1%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/stream-cache-hit
Rule: Input tokens written to the provider cache.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--stream-cache-hit/v1/messages`

chat claude-sonnet-5-5 openlit.instrumentation.anthropic, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`2054`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stop_sequences`	`[]`
`gen_ai.request.stream`	`True`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_k`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.response.finish_reasons`	`['end_turn']`
`gen_ai.response.id`	`msg_wt_scache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.sdk.version`	`0.125.0`
`gen_ai.server.time_per_output_token`	`0.0010051727294921875`
`gen_ai.server.time_to_first_token`	`0.005663871765136719`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`1543`
`gen_ai.usage.output_tokens`	`511`
`openlit.agent.version_hash`	`126a1732f5faa9e0`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · anthropic/stream-dropped · span.endedGhost span #
never ended: chat claude-sonnet-5-5

Note: never ended: chat claude-sonnet-5-5
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/stream-dropped
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--stream-dropped/v1/messages`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · anthropic/stream-early-exit · span.endedGhost span #
never ended: chat claude-sonnet-5-5

Note: never ended: chat claude-sonnet-5-5
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/stream-early-exit
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--stream-early-exit/v1/messages`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · anthropic/stream-early-exit-with · span.endedGhost span #
never ended: chat claude-sonnet-5-5

Note: never ended: chat claude-sonnet-5-5
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/stream-early-exit-with
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--stream-early-exit-with/v1/messages`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · anthropic/stream-error-event · span.endedGhost span #
never ended: chat claude-sonnet-5-5

Note: never ended: chat claude-sonnet-5-5
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario anthropic/stream-error-event
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--stream-error-event/v1/messages`

spantruth.scenario spantruth.driver, status UNSET

missing openlit · anthropic/stream-thinking · reasoning_tokensHidden reasoning #
expected 1103, reported nothing

Expected: 1103
Reproduce: spantruth run --subject openlit --scenario anthropic/stream-thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--stream-thinking/v1/messages`

chat claude-sonnet-5-5 openlit.instrumentation.anthropic, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`3270`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stop_sequences`	`[]`
`gen_ai.request.stream`	`True`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_k`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.response.finish_reasons`	`['end_turn']`
`gen_ai.response.id`	`msg_wt_sthink`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.sdk.version`	`0.125.0`
`gen_ai.server.time_per_output_token`	`0.0009129780989426833`
`gen_ai.server.time_to_first_token`	`0.008232355117797852`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1439`
`openlit.agent.version_hash`	`126a1732f5faa9e0`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

missing openlit · anthropic/thinking · reasoning_tokensHidden reasoning #
expected 947, reported nothing

Expected: 947
Reproduce: spantruth run --subject openlit --scenario anthropic/thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/anthropic--thinking/v1/messages`

chat claude-sonnet-5-5 openlit.instrumentation.anthropic, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`2900`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stop_sequences`	`[]`
`gen_ai.request.stream`	`False`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_k`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.response.finish_reasons`	`['end_turn']`
`gen_ai.response.id`	`msg_wt_thinking`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.sdk.version`	`0.125.0`
`gen_ai.server.time_per_output_token`	`0`
`gen_ai.server.time_to_first_token`	`0.013167619705200195`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`
`openlit.agent.version_hash`	`126a1732f5faa9e0`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · openai/async-stream-dropped · span.endedGhost span #
never ended: chat gpt-5-mini

Note: never ended: chat gpt-5-mini
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario openai/async-stream-dropped
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--async-stream-dropped/v1/chat/completions`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · openai/async-stream-early-exit · span.endedGhost span #
never ended: chat gpt-5-mini

Note: never ended: chat gpt-5-mini
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario openai/async-stream-early-exit
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--async-stream-early-exit/v1/chat/completions`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · openai/async-stream-helper · span.endedGhost span #
never ended: chat gpt-5-mini

Note: never ended: chat gpt-5-mini
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario openai/async-stream-helper
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--async-stream-helper/v1/chat/completions`

spantruth.scenario spantruth.driver, status UNSET

missing openlit · openai/reasoning · reasoning_tokensHidden reasoning #
expected 1792, reported nothing

Expected: 1792
Reproduce: spantruth run --subject openlit --scenario openai/reasoning
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--reasoning/v1/chat/completions`

chat gpt-5-mini openlit.instrumentation.openai, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`3820`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`openai`
`gen_ai.request.frequency_penalty`	`0.0`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.request.presence_penalty`	`0.0`
`gen_ai.request.seed`	`0`
`gen_ai.request.stream`	`False`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.request.user`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-reason`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.sdk.version`	`2.54.0`
`gen_ai.server.time_per_output_token`	`0`
`gen_ai.server.time_to_first_token`	`0.018053054809570312`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`1601`
`gen_ai.usage.output_tokens`	`2219`
`openai.api.type`	`chat_completions`
`openai.response.service_tier`	`default`
`openai.response.system_fingerprint`	`fp_wt0001`
`openlit.agent.version_hash`	`8559be562eb7885f`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · openai/responses-stream-missing-finish-abort · span.endedGhost span #
never ended: chat gpt-6-luna

Note: never ended: chat gpt-6-luna
Reproduce: spantruth run --subject openlit --scenario openai/responses-stream-missing-finish-abort
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--responses-stream-missing-finish-abort/v1/responses`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · openai/stream-dropped · span.endedGhost span #
never ended: chat gpt-5-mini

Note: never ended: chat gpt-5-mini
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario openai/stream-dropped
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--stream-dropped/v1/chat/completions`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · openai/stream-early-exit · span.endedGhost span #
never ended: chat gpt-5-mini

Note: never ended: chat gpt-5-mini
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario openai/stream-early-exit
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--stream-early-exit/v1/chat/completions`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · openai/stream-early-exit-with · span.endedGhost span #
never ended: chat gpt-5-mini

Note: never ended: chat gpt-5-mini
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario openai/stream-early-exit-with
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--stream-early-exit-with/v1/chat/completions`

spantruth.scenario spantruth.driver, status UNSET

lost openlit · openai/stream-helper · span.endedGhost span #
never ended: chat gpt-5-mini

Note: never ended: chat gpt-5-mini
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openlit --scenario openai/stream-helper
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (2)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--stream-helper/v1/chat/completions`

spantruth.scenario spantruth.driver, status UNSET

wrong openlit · openai/stream-no-usage · input_tokensPhantom tokens #
expected nothing, reported 18: estimated: the stream carried no usage

Reported: 18
Read from: gen_ai.usage.input_tokens
Note: estimated: the stream carried no usage
Reproduce: spantruth run --subject openlit --scenario openai/stream-no-usage
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--stream-no-usage/v1/chat/completions`

chat gpt-5-mini openlit.instrumentation.openai, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`34`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`openai`
`gen_ai.request.frequency_penalty`	`0.0`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.request.presence_penalty`	`0.0`
`gen_ai.request.seed`	`0`
`gen_ai.request.stream`	`True`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.request.user`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-nousage`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.sdk.version`	`2.54.0`
`gen_ai.server.time_per_output_token`	`0.00015342235565185547`
`gen_ai.server.time_to_first_token`	`0.002485036849975586`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`18`
`gen_ai.usage.output_tokens`	`16`
`openai.api.type`	`chat_completions`
`openai.response.service_tier`	`default`
`openai.response.system_fingerprint`	`fp_wt0001`
`openlit.agent.version_hash`	`8559be562eb7885f`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

wrong openlit · openai/stream-no-usage · output_tokensPhantom tokens #
expected nothing, reported 16: estimated: the stream carried no usage

Reported: 16
Read from: gen_ai.usage.output_tokens
Note: estimated: the stream carried no usage
Reproduce: spantruth run --subject openlit --scenario openai/stream-no-usage
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--stream-no-usage/v1/chat/completions`

chat gpt-5-mini openlit.instrumentation.openai, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`34`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`openai`
`gen_ai.request.frequency_penalty`	`0.0`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.request.presence_penalty`	`0.0`
`gen_ai.request.seed`	`0`
`gen_ai.request.stream`	`True`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.request.user`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-nousage`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.sdk.version`	`2.54.0`
`gen_ai.server.time_per_output_token`	`0.00015342235565185547`
`gen_ai.server.time_to_first_token`	`0.002485036849975586`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`18`
`gen_ai.usage.output_tokens`	`16`
`openai.api.type`	`chat_completions`
`openai.response.service_tier`	`default`
`openai.response.system_fingerprint`	`fp_wt0001`
`openlit.agent.version_hash`	`8559be562eb7885f`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

missing openlit · openai/stream-reasoning · reasoning_tokensHidden reasoning #
expected 1931, reported nothing

Expected: 1931
Reproduce: spantruth run --subject openlit --scenario openai/stream-reasoning
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (3)

POST opentelemetry.instrumentation.httpx, status UNSET

`http.method`	`POST`
`http.status_code`	`200`
`http.url`	`http://127.0.0.1:46433/s/openai--stream-reasoning/v1/chat/completions`

chat gpt-5-mini openlit.instrumentation.openai, status OK

`deployment.environment`	`default`
`gen_ai.client.token.usage`	`4548`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.output.type`	`text`
`gen_ai.provider.name`	`openai`
`gen_ai.request.frequency_penalty`	`0.0`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.request.presence_penalty`	`0.0`
`gen_ai.request.seed`	`0`
`gen_ai.request.stream`	`True`
`gen_ai.request.temperature`	`1.0`
`gen_ai.request.top_p`	`1.0`
`gen_ai.request.user`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-sreason`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.sdk.version`	`2.54.0`
`gen_ai.server.time_per_output_token`	`0.0002781867980957031`
`gen_ai.server.time_to_first_token`	`0.0029630661010742188`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cost`	`0`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`openai.api.type`	`chat_completions`
`openai.response.service_tier`	`default`
`openai.response.system_fingerprint`	`fp_wt0001`
`openlit.agent.version_hash`	`8559be562eb7885f`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`service.name`	`spantruth`
`telemetry.sdk.name`	`openlit`

spantruth.scenario spantruth.driver, status UNSET

wrong openllmetry · anthropic/async-stream · output_tokensPhantom tokens #
expected 419, reported 420

Expected: 419
Reported: 420
Read from: gen_ai.usage.output_tokens
Upstream: reported message_delta carries the cumulative output count, and the wrapper adds it to message_start's placeholder of 1.
Cost: $0.00881 on the wire, $0.00882 from the spans (+0.1%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/async-stream
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_stream`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`420`
`gen_ai.usage.total_tokens`	`2731`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · anthropic/async-stream-dropped · error.typeFalse OK #

Reproduce: spantruth run --subject openllmetry --scenario anthropic/async-stream-dropped
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status ERROR

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

lost openllmetry · anthropic/async-stream-early-exit · span.endedGhost span #
never ended: anthropic.chat

Note: never ended: anthropic.chat
Upstream: reported Fix proposed in https://github.com/traceloop/openllmetry/pull/4539.
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/async-stream-early-exit
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · anthropic/async-stream-error-event · error.typeFalse OK #

Reproduce: spantruth run --subject openllmetry --scenario anthropic/async-stream-error-event
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status ERROR

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`

spantruth.scenario spantruth.driver, status UNSET

wrong openllmetry · anthropic/stream · output_tokensPhantom tokens #
expected 419, reported 420

Expected: 419
Reported: 420
Read from: gen_ai.usage.output_tokens
Upstream: reported message_delta carries the cumulative output count, and the wrapper adds it to message_start's placeholder of 1.
Cost: $0.00881 on the wire, $0.00882 from the spans (+0.1%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_stream`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`420`
`gen_ai.usage.total_tokens`	`2731`

spantruth.scenario spantruth.driver, status UNSET

wrong openllmetry · anthropic/stream-cache-cumulative · output_tokensPhantom tokens #
expected 467, reported 468

Expected: 467
Reported: 468
Read from: gen_ai.usage.output_tokens
Upstream: reported message_delta carries the cumulative output count, and the wrapper adds it to message_start's placeholder of 1.
Cost: $0.00961 on the wire, $0.00962 from the spans (+0.1%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream-cache-cumulative
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_cumul`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_creation.input_tokens`	`811`
`gen_ai.usage.cache_read.input_tokens`	`3583`
`gen_ai.usage.input_tokens`	`5673`
`gen_ai.usage.output_tokens`	`468`
`gen_ai.usage.total_tokens`	`6141`

spantruth.scenario spantruth.driver, status UNSET

wrong openllmetry · anthropic/stream-cache-hit · output_tokensPhantom tokens #
expected 511, reported 512

Expected: 511
Reported: 512
Read from: gen_ai.usage.output_tokens
Upstream: reported message_delta carries the cumulative output count, and the wrapper adds it to message_start's placeholder of 1.
Cost: $0.0107 on the wire, $0.0107 from the spans (+0.1%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream-cache-hit
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_scache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_creation.input_tokens`	`737`
`gen_ai.usage.cache_read.input_tokens`	`6144`
`gen_ai.usage.input_tokens`	`8424`
`gen_ai.usage.output_tokens`	`512`
`gen_ai.usage.total_tokens`	`8936`

spantruth.scenario spantruth.driver, status UNSET

lost openllmetry · anthropic/stream-dropped · span.endedGhost span #
never ended: anthropic.chat

Note: never ended: anthropic.chat
Upstream: reported The async stream ends its span with status ERROR, but without error.type.
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream-dropped
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

lost openllmetry · anthropic/stream-early-exit · span.endedGhost span #
never ended: anthropic.chat

Note: never ended: anthropic.chat
Upstream: reported Fix proposed in https://github.com/traceloop/openllmetry/pull/4539.
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream-early-exit
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

lost openllmetry · anthropic/stream-early-exit-with · span.endedGhost span #
never ended: anthropic.chat

Note: never ended: anthropic.chat
Upstream: reported Fix proposed in https://github.com/traceloop/openllmetry/pull/4539.
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream-early-exit-with
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

lost openllmetry · anthropic/stream-error-event · span.endedGhost span #
never ended: anthropic.chat

Note: never ended: anthropic.chat
Upstream: reported The async stream ends its span with status ERROR, but without error.type.
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream-error-event
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

wrong openllmetry · anthropic/stream-thinking · output_tokensPhantom tokens #
expected 1439, reported 1440

Expected: 1439
Reported: 1440
Read from: gen_ai.usage.output_tokens
Cost: $0.0181 on the wire, $0.0181 from the spans (+0.1%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream-thinking
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "reasoning", "content": "Let me think about the capital of France. The capital is Paris."}, {"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_sthink`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1440`
`gen_ai.usage.total_tokens`	`3271`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · anthropic/stream-thinking · reasoning_tokensHidden reasoning #
expected 1103, reported nothing

Expected: 1103
Cost: $0.0181 on the wire, $0.0181 from the spans (+0.1%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream-thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "reasoning", "content": "Let me think about the capital of France. The capital is Paris."}, {"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_sthink`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1440`
`gen_ai.usage.total_tokens`	`3271`

spantruth.scenario spantruth.driver, status UNSET

wrong openllmetry · anthropic/stream-tool-call · output_tokensPhantom tokens #
expected 197, reported 198

Expected: 197
Reported: 198
Read from: gen_ai.usage.output_tokens
Cost: $0.00521 on the wire, $0.00522 from the spans (+0.2%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario anthropic/stream-tool-call
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "tool_call", "id": "toolu_wt_1", "name": "get_weather", "arguments": {"city": "Paris"}}, {"type": "tool_call", "id": "toolu_wt_2", "name": "get_time", "arguments": {"zone": "Europe/Paris"}}], "finish_reason": "tool_call"}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['tool_call']`
`gen_ai.response.id`	`msg_wt_stools`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.tool.definitions`	`[{"name": "get_weather", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"name": "get_time", "input_schema": {"type": "object", "properties": {"zone": {"type": "string"}}}}]`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1619`
`gen_ai.usage.output_tokens`	`198`
`gen_ai.usage.total_tokens`	`1817`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · anthropic/thinking · reasoning_tokensHidden reasoning #
expected 947, reported nothing

Expected: 947
Reproduce: spantruth run --subject openllmetry --scenario anthropic/thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (2)

anthropic.chat opentelemetry.instrumentation.anthropic, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"type": "text", "content": "What is the capital of France?"}]}]`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"type": "reasoning", "content": "Let me think about the capital of France. The capital of France is Paris."}, {"type": "text", "content": "The capital of France is Paris."}], "finish_reason": "stop"}]`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_thinking`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_creation.input_tokens`	`0`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`
`gen_ai.usage.total_tokens`	`2900`

spantruth.scenario spantruth.driver, status UNSET

wrong openllmetry · openai/async-stream-dropped · error.statusFalse OK #
expected ERROR, reported OK

Expected: ERROR
Reported: ['OK']
Upstream: reported _ensure_cleanup() sets OK and ends the span before the ERROR status is set, so ERROR is dropped.
Reproduce: spantruth run --subject openllmetry --scenario openai/async-stream-dropped
Rule: Span status SHOULD follow Recording Errors: an operation that ended in an error sets the status to Error.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--async-stream-dropped/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"content": "The capital of France", "type": "text"}], "finish_reason": ""}]`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.id`	`chatcmpl-wt-drop`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/async-stream-dropped · error.typeFalse OK #

Reproduce: spantruth run --subject openllmetry --scenario openai/async-stream-dropped
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--async-stream-dropped/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"content": "The capital of France", "type": "text"}], "finish_reason": ""}]`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.id`	`chatcmpl-wt-drop`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/async-stream-helper · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/async-stream-helper
Rule: The name of the model that generated the response.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--async-stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/async-stream-helper · response_id #
expected chatcmpl-wt-shelper, reported nothing

Expected: chatcmpl-wt-shelper
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/async-stream-helper
Rule: The unique identifier of the completion.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--async-stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/async-stream-helper · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/async-stream-helper
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--async-stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/async-stream-helper · input_tokensVanishing tokens #
expected 2137, reported nothing

Expected: 2137
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/async-stream-helper
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--async-stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/async-stream-helper · output_tokensVanishing tokens #
expected 313, reported nothing

Expected: 313
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/async-stream-helper
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--async-stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

wrong openllmetry · openai/stream-dropped · error.statusFalse OK #
expected ERROR, reported OK

Expected: ERROR
Reported: ['OK']
Upstream: reported _ensure_cleanup() sets OK and ends the span before the ERROR status is set, so ERROR is dropped.
Reproduce: spantruth run --subject openllmetry --scenario openai/stream-dropped
Rule: Span status SHOULD follow Recording Errors: an operation that ended in an error sets the status to Error.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--stream-dropped/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"content": "The capital of France", "type": "text"}], "finish_reason": ""}]`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.id`	`chatcmpl-wt-drop`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/stream-dropped · error.typeFalse OK #

Reproduce: spantruth run --subject openllmetry --scenario openai/stream-dropped
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--stream-dropped/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"content": "The capital of France", "type": "text"}], "finish_reason": ""}]`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.id`	`chatcmpl-wt-drop`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/stream-helper · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/stream-helper
Rule: The name of the model that generated the response.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/stream-helper · response_id #
expected chatcmpl-wt-shelper, reported nothing

Expected: chatcmpl-wt-shelper
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/stream-helper
Rule: The unique identifier of the completion.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/stream-helper · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/stream-helper
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/stream-helper · input_tokensVanishing tokens #
expected 2137, reported nothing

Expected: 2137
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/stream-helper
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/stream-helper · output_tokensVanishing tokens #
expected 313, reported nothing

Expected: 313
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject openllmetry --scenario openai/stream-helper
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status UNSET

`gen_ai.headers`	`{'X-Stainless-Raw-Response': 'stream'}`
`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--stream-helper/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`

spantruth.scenario spantruth.driver, status UNSET

missing openllmetry · openai/stream-reasoning · reasoning_tokensHidden reasoning #
expected 1931, reported nothing

Expected: 1931
Reproduce: spantruth run --subject openllmetry --scenario openai/stream-reasoning
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (2)

openai.chat opentelemetry.instrumentation.openai.v1, status OK

`gen_ai.input.messages`	`[{"role": "user", "parts": [{"content": "What is the capital of France?", "type": "text"}]}]`
`gen_ai.is_streaming`	`True`
`gen_ai.openai.api_base`	`http://127.0.0.1:46433/s/openai--stream-reasoning/v1/`
`gen_ai.operation.name`	`chat`
`gen_ai.output.messages`	`[{"role": "assistant", "parts": [{"content": "The capital of France is Paris.", "type": "text"}], "finish_reason": "stop"}]`
`gen_ai.provider.name`	`openai`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`chatcmpl-wt-sreason`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`gen_ai.usage.total_tokens`	`4548`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/async-basic · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.005 on the wire, $0.01 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/async-basic
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_basic`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--async-basic/v1/messages`
`url.path`	`/s/anthropic--async-basic/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_basic`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/async-stream · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.00881 on the wire, $0.0176 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/async-stream
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_stream`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.067773666`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--async-stream/v1/messages`
`url.path`	`/s/anthropic--async-stream/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_stream`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.06517009899994264`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/async-stream-early-exit · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/async-stream-early-exit
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_early`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1913`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--async-stream-early-exit/v1/messages`
`url.path`	`/s/anthropic--async-stream-early-exit/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_early`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.07421174799992514`
`gen_ai.usage.input_tokens`	`1913`
`gen_ai.usage.output_tokens`	`1`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/basic · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.005 on the wire, $0.01 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/basic
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_basic`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--basic/v1/messages`
`url.path`	`/s/anthropic--basic/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_basic`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/cache-hit · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.00785 on the wire, $0.0157 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/cache-hit
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_cache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`4096`
`gen_ai.usage.cache_write.input_tokens`	`519`
`gen_ai.usage.input_tokens`	`5802`
`gen_ai.usage.output_tokens`	`377`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--cache-hit/v1/messages`
`url.path`	`/s/anthropic--cache-hit/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_cache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`4096`
`gen_ai.usage.cache_write.input_tokens`	`519`
`gen_ai.usage.input_tokens`	`5802`
`gen_ai.usage.output_tokens`	`377`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/retry-429 · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.00862 on the wire, $0.0172 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/retry-429
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_retry`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`http.request.method`	`POST`
`http.request.resend_count`	`1`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--retry-429/v1/messages`
`url.path`	`/s/anthropic--retry-429/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_retry`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/stream · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.00881 on the wire, $0.0176 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/stream
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_stream`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.029754198`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream/v1/messages`
`url.path`	`/s/anthropic--stream/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_stream`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.025595997999971587`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/stream-cache-cumulative · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.00961 on the wire, $0.0192 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/stream-cache-cumulative
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_cumul`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.029513531`
`gen_ai.usage.cache_read.input_tokens`	`3583`
`gen_ai.usage.cache_write.input_tokens`	`811`
`gen_ai.usage.input_tokens`	`5673`
`gen_ai.usage.output_tokens`	`467`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-cache-cumulative/v1/messages`
`url.path`	`/s/anthropic--stream-cache-cumulative/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_cumul`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.02534868899999765`
`gen_ai.usage.cache_read.input_tokens`	`3583`
`gen_ai.usage.cache_write.input_tokens`	`811`
`gen_ai.usage.input_tokens`	`5673`
`gen_ai.usage.output_tokens`	`467`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/stream-cache-hit · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.0107 on the wire, $0.0213 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/stream-cache-hit
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_scache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.029889572`
`gen_ai.usage.cache_read.input_tokens`	`6144`
`gen_ai.usage.cache_write.input_tokens`	`737`
`gen_ai.usage.input_tokens`	`8424`
`gen_ai.usage.output_tokens`	`511`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-cache-hit/v1/messages`
`url.path`	`/s/anthropic--stream-cache-hit/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_scache`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.025751530000093226`
`gen_ai.usage.cache_read.input_tokens`	`6144`
`gen_ai.usage.cache_write.input_tokens`	`737`
`gen_ai.usage.input_tokens`	`8424`
`gen_ai.usage.output_tokens`	`511`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/stream-early-exit · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/stream-early-exit
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_early`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1913`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-early-exit/v1/messages`
`url.path`	`/s/anthropic--stream-early-exit/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_early`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.024970992000021397`
`gen_ai.usage.input_tokens`	`1913`
`gen_ai.usage.output_tokens`	`1`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/stream-early-exit-with · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/stream-early-exit-with
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_early`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1913`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-early-exit-with/v1/messages`
`url.path`	`/s/anthropic--stream-early-exit-with/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_early`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.022960990000001402`
`gen_ai.usage.input_tokens`	`1913`
`gen_ai.usage.output_tokens`	`1`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/stream-thinking · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.0181 on the wire, $0.0361 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/stream-thinking
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_sthink`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.029223`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1439`
`gen_ai.usage.reasoning.output_tokens`	`1103`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-thinking/v1/messages`
`url.path`	`/s/anthropic--stream-thinking/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_sthink`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.02548620800007484`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.output_tokens`	`1439`
`gen_ai.usage.reasoning.output_tokens`	`1103`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/stream-tool-call · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.00521 on the wire, $0.0104 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/stream-tool-call
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`tool_use`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.id`	`msg_wt_stools`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.03069204`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1619`
`gen_ai.usage.output_tokens`	`197`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--stream-tool-call/v1/messages`
`url.path`	`/s/anthropic--stream-tool-call/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.request.stream`	`True`
`gen_ai.response.finish_reasons`	`['tool_call']`
`gen_ai.response.id`	`msg_wt_stools`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.time_to_first_chunk`	`0.0268505640000285`
`gen_ai.usage.input_tokens`	`1619`
`gen_ai.usage.output_tokens`	`197`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/thinking · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.016 on the wire, $0.032 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/thinking
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`end_turn`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_thinking`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`
`gen_ai.usage.reasoning.output_tokens`	`947`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--thinking/v1/messages`
`url.path`	`/s/anthropic--thinking/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['stop']`
`gen_ai.response.id`	`msg_wt_thinking`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.output_tokens`	`1273`
`gen_ai.usage.reasoning.output_tokens`	`947`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

wrong otel-official-sdk-on · anthropic/tool-call · tokens.single-reportDouble span #
expected 1, reported 2: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5

Expected: 1
Reported: 2
Note: usage on 2 spans: anthropic.messages.create, chat claude-sonnet-5-5
Cost: $0.00274 on the wire, $0.00548 from the spans (+100%), at list price.
Reproduce: spantruth run --subject otel-official-sdk-on --scenario anthropic/tool-call
Rule: spantruth rule: one call reports its token usage on one span. Usage on two spans of one call is counted twice by any backend that sums token attributes across spans.

Exported spans (3)

anthropic.messages.create com.anthropic.sdk.python, status OK

`anthropic.message.stop_reason`	`tool_use`
`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_tools`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.cache_read.input_tokens`	`0`
`gen_ai.usage.cache_write.input_tokens`	`0`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`http.request.method`	`POST`
`http.response.status_code`	`200`
`http.route`	`/v1/messages`
`server.address`	`127.0.0.1`
`server.port`	`46433`
`url.full`	`http://127.0.0.1:46433/s/anthropic--tool-call/v1/messages`
`url.path`	`/s/anthropic--tool-call/v1/messages`
`url.scheme`	`http`

chat claude-sonnet-5-5 opentelemetry.instrumentation.genai.anthropic, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.provider.name`	`anthropic`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['tool_call']`
`gen_ai.response.id`	`msg_wt_tools`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`server.address`	`127.0.0.1`
`server.port`	`46433`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · anthropic/async-stream-dropped · error.typeFalse OK #

Reproduce: spantruth run --subject sentry --scenario anthropic/async-stream-dropped
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

chat claude-sonnet-5-5 sentry_sdk.adapter, status ERROR

`gen_ai.operation.name`	`chat`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_drop`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`anthropic`
`gen_ai.usage.input_tokens`	`1427`
`gen_ai.usage.input_tokens.cache_write`	`0`
`gen_ai.usage.input_tokens.cached`	`0`
`gen_ai.usage.output_tokens`	`1`
`gen_ai.usage.total_tokens`	`1428`
`thread.id`	`124264826677120`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · anthropic/async-stream-error-event · error.typeFalse OK #

Reproduce: spantruth run --subject sentry --scenario anthropic/async-stream-error-event
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

chat claude-sonnet-5-5 sentry_sdk.adapter, status ERROR

`gen_ai.operation.name`	`chat`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_overload`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`anthropic`
`gen_ai.usage.input_tokens`	`1301`
`gen_ai.usage.input_tokens.cache_write`	`0`
`gen_ai.usage.input_tokens.cached`	`0`
`gen_ai.usage.output_tokens`	`1`
`gen_ai.usage.total_tokens`	`1302`
`thread.id`	`129641097131904`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · anthropic/rate-limited · error.typeFalse OK #

Reproduce: spantruth run --subject sentry --scenario anthropic/rate-limited
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

chat claude-sonnet-5-5 sentry_sdk.adapter, status ERROR

`gen_ai.operation.name`	`chat`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`anthropic`
`thread.id`	`129242343815040`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · anthropic/stream-dropped · error.typeFalse OK #

Reproduce: spantruth run --subject sentry --scenario anthropic/stream-dropped
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

chat claude-sonnet-5-5 sentry_sdk.adapter, status ERROR

`gen_ai.operation.name`	`chat`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_drop`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`anthropic`
`gen_ai.usage.input_tokens`	`1427`
`gen_ai.usage.input_tokens.cache_write`	`0`
`gen_ai.usage.input_tokens.cached`	`0`
`gen_ai.usage.output_tokens`	`1`
`gen_ai.usage.total_tokens`	`1428`
`thread.id`	`129569111238528`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · anthropic/stream-error-event · error.typeFalse OK #

Reproduce: spantruth run --subject sentry --scenario anthropic/stream-error-event
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

chat claude-sonnet-5-5 sentry_sdk.adapter, status ERROR

`gen_ai.operation.name`	`chat`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.id`	`msg_wt_overload`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`anthropic`
`gen_ai.usage.input_tokens`	`1301`
`gen_ai.usage.input_tokens.cache_write`	`0`
`gen_ai.usage.input_tokens.cached`	`0`
`gen_ai.usage.output_tokens`	`1`
`gen_ai.usage.total_tokens`	`1302`
`thread.id`	`124202038421376`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · anthropic/stream-thinking · reasoning_tokensHidden reasoning #
expected 1103, reported nothing

Expected: 1103
Reproduce: spantruth run --subject sentry --scenario anthropic/stream-thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (2)

chat claude-sonnet-5-5 sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['end_turn']`
`gen_ai.response.id`	`msg_wt_sthink`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`anthropic`
`gen_ai.usage.input_tokens`	`1831`
`gen_ai.usage.input_tokens.cache_write`	`0`
`gen_ai.usage.input_tokens.cached`	`0`
`gen_ai.usage.output_tokens`	`1439`
`gen_ai.usage.total_tokens`	`3270`
`thread.id`	`135552638535552`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · anthropic/thinking · reasoning_tokensHidden reasoning #
expected 947, reported nothing

Expected: 947
Reproduce: spantruth run --subject sentry --scenario anthropic/thinking
Rule: Output tokens used for reasoning. For Anthropic, thinking tokens reported in `output_tokens_details` map to this count. OpenInference records them as `llm.token_count.completion_details.reasoning`.

Exported spans (2)

chat claude-sonnet-5-5 sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.max_tokens`	`1024`
`gen_ai.request.model`	`claude-sonnet-5-5`
`gen_ai.response.finish_reasons`	`['end_turn']`
`gen_ai.response.id`	`msg_wt_thinking`
`gen_ai.response.model`	`claude-sonnet-5-5-20260915`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`anthropic`
`gen_ai.usage.input_tokens`	`1627`
`gen_ai.usage.input_tokens.cache_write`	`0`
`gen_ai.usage.input_tokens.cached`	`0`
`gen_ai.usage.output_tokens`	`1273`
`gen_ai.usage.total_tokens`	`2900`
`thread.id`	`129514418210688`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/async-basic · response_id #
expected chatcmpl-wt-basic, reported nothing

Expected: chatcmpl-wt-basic
Reproduce: spantruth run --subject sentry --scenario openai/async-basic
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`thread.id`	`140207012571008`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/async-basic · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/async-basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`thread.id`	`140207012571008`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/async-stream · response_id #
expected chatcmpl-wt-stream, reported nothing

Expected: chatcmpl-wt-stream
Reproduce: spantruth run --subject sentry --scenario openai/async-stream
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.054270317000032264`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`thread.id`	`139437739747200`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/async-stream · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/async-stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.054270317000032264`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`thread.id`	`139437739747200`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

lost sentry · openai/async-stream-dropped · span.endedGhost span #
never ended: gen_ai.chat

Note: never ended: gen_ai.chat
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/async-stream-dropped
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

lost sentry · openai/async-stream-early-exit · span.endedGhost span #
never ended: gen_ai.chat

Note: never ended: gen_ai.chat
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/async-stream-early-exit
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/async-stream-helper · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/async-stream-helper
Rule: The name of the model that generated the response.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`132676248496000`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/async-stream-helper · response_id #
expected chatcmpl-wt-shelper, reported nothing

Expected: chatcmpl-wt-shelper
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/async-stream-helper
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`132676248496000`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/async-stream-helper · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/async-stream-helper
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`132676248496000`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/async-stream-helper · input_tokensVanishing tokens #
expected 2137, reported nothing

Expected: 2137
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/async-stream-helper
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`132676248496000`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/async-stream-helper · output_tokensVanishing tokens #
expected 313, reported nothing

Expected: 313
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/async-stream-helper
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`132676248496000`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/basic · response_id #
expected chatcmpl-wt-basic, reported nothing

Expected: chatcmpl-wt-basic
Reproduce: spantruth run --subject sentry --scenario openai/basic
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`thread.id`	`132046691985280`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/basic · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/basic
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`1187`
`gen_ai.usage.output_tokens`	`263`
`gen_ai.usage.total_tokens`	`1450`
`thread.id`	`132046691985280`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/cache-hit · response_id #
expected chatcmpl-wt-cache, reported nothing

Expected: chatcmpl-wt-cache
Reproduce: spantruth run --subject sentry --scenario openai/cache-hit
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`5379`
`gen_ai.usage.input_tokens.cached`	`4096`
`gen_ai.usage.output_tokens`	`377`
`gen_ai.usage.total_tokens`	`5756`
`thread.id`	`128849767057280`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/cache-hit · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`5379`
`gen_ai.usage.input_tokens.cached`	`4096`
`gen_ai.usage.output_tokens`	`377`
`gen_ai.usage.total_tokens`	`5756`
`thread.id`	`128849767057280`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/rate-limited · error.typeFalse OK #

Reproduce: spantruth run --subject sentry --scenario openai/rate-limited
Rule: `error.type` is conditionally required if the operation ended in an error. Any low-cardinality identifier is allowed, so only its presence is graded.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status ERROR

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`thread.id`	`127064647653248`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/reasoning · response_id #
expected chatcmpl-wt-reason, reported nothing

Expected: chatcmpl-wt-reason
Reproduce: spantruth run --subject sentry --scenario openai/reasoning
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`1601`
`gen_ai.usage.output_tokens`	`2219`
`gen_ai.usage.output_tokens.reasoning`	`1792`
`gen_ai.usage.total_tokens`	`3820`
`thread.id`	`127834311047040`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/reasoning · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/reasoning
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`1601`
`gen_ai.usage.output_tokens`	`2219`
`gen_ai.usage.output_tokens.reasoning`	`1792`
`gen_ai.usage.total_tokens`	`3820`
`thread.id`	`127834311047040`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

lost sentry · openai/responses-stream-missing-finish-abort · span.endedGhost span #
never ended: gen_ai.responses

Note: never ended: gen_ai.responses
Reproduce: spantruth run --subject sentry --scenario openai/responses-stream-missing-finish-abort
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/retry-429 · response_id #
expected chatcmpl-wt-retry, reported nothing

Expected: chatcmpl-wt-retry
Reproduce: spantruth run --subject sentry --scenario openai/retry-429
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`gen_ai.usage.total_tokens`	`2844`
`thread.id`	`140402769287040`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/retry-429 · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/retry-429
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`2477`
`gen_ai.usage.output_tokens`	`367`
`gen_ai.usage.total_tokens`	`2844`
`thread.id`	`140402769287040`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream · response_id #
expected chatcmpl-wt-stream, reported nothing

Expected: chatcmpl-wt-stream
Reproduce: spantruth run --subject sentry --scenario openai/stream
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.013695299999994859`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`thread.id`	`138128027133824`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/stream
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.013695299999994859`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`2311`
`gen_ai.usage.output_tokens`	`419`
`gen_ai.usage.total_tokens`	`2730`
`thread.id`	`138128027133824`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-cache-hit · response_id #
expected chatcmpl-wt-scache, reported nothing

Expected: chatcmpl-wt-scache
Reproduce: spantruth run --subject sentry --scenario openai/stream-cache-hit
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.008735299000022678`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`7213`
`gen_ai.usage.input_tokens.cached`	`6144`
`gen_ai.usage.output_tokens`	`511`
`gen_ai.usage.total_tokens`	`7724`
`thread.id`	`131498694985600`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-cache-hit · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/stream-cache-hit
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.008735299000022678`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`7213`
`gen_ai.usage.input_tokens.cached`	`6144`
`gen_ai.usage.output_tokens`	`511`
`gen_ai.usage.total_tokens`	`7724`
`thread.id`	`131498694985600`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

lost sentry · openai/stream-dropped · span.endedGhost span #
never ended: gen_ai.chat

Note: never ended: gen_ai.chat
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/stream-dropped
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

lost sentry · openai/stream-early-exit · span.endedGhost span #
never ended: gen_ai.chat

Note: never ended: gen_ai.chat
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/stream-early-exit
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

lost sentry · openai/stream-early-exit-with · span.endedGhost span #
never ended: gen_ai.chat

Note: never ended: gen_ai.chat
Cost: Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/stream-early-exit-with
Rule: GenAI spans SHOULD end when the response is fully received or the operation is terminated due to an error or cancellation.

Exported spans (1)

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-helper · response_model #
expected gpt-5-mini-2026-08-07, reported nothing

Expected: gpt-5-mini-2026-08-07
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/stream-helper
Rule: The name of the model that generated the response.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`129915340576640`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-helper · response_id #
expected chatcmpl-wt-shelper, reported nothing

Expected: chatcmpl-wt-shelper
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/stream-helper
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`129915340576640`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-helper · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/stream-helper
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`129915340576640`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-helper · input_tokensVanishing tokens #
expected 2137, reported nothing

Expected: 2137
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/stream-helper
Rule: Input tokens SHOULD include all types of input tokens, including cached tokens. For Anthropic, whose `input_tokens` excludes them, cache reads and cache writes MUST be added. OpenInference states the same rule for `llm.token_count.prompt`.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`129915340576640`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-helper · output_tokensVanishing tokens #
expected 313, reported nothing

Expected: 313
Cost: $0.00116 on the wire. Nothing reaches a backend (-100%), at list price.
Reproduce: spantruth run --subject sentry --scenario openai/stream-helper
Rule: Output tokens, with reasoning tokens included. For Anthropic, whose `output_tokens` includes thinking tokens, reasoning tokens are part of, not in addition to, output tokens. OpenInference states the same rule for `llm.token_count.completion`.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.streaming`	`True`
`gen_ai.system`	`openai`
`thread.id`	`129915340576640`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-no-usage · response_id #
expected chatcmpl-wt-nousage, reported nothing

Expected: chatcmpl-wt-nousage
Reproduce: spantruth run --subject sentry --scenario openai/stream-no-usage
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.013003673999946841`
`gen_ai.system`	`openai`
`thread.id`	`129330600253312`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-no-usage · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/stream-no-usage
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.013003673999946841`
`gen_ai.system`	`openai`
`thread.id`	`129330600253312`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-reasoning · response_id #
expected chatcmpl-wt-sreason, reported nothing

Expected: chatcmpl-wt-sreason
Reproduce: spantruth run --subject sentry --scenario openai/stream-reasoning
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.01354161100005058`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`gen_ai.usage.output_tokens.reasoning`	`1931`
`gen_ai.usage.total_tokens`	`4548`
`thread.id`	`132539277981568`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-reasoning · finish_reasonsSilent stop #
expected stop, reported nothing

Expected: ['stop']
Reproduce: spantruth run --subject sentry --scenario openai/stream-reasoning
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.01354161100005058`
`gen_ai.system`	`openai`
`gen_ai.usage.input_tokens`	`2089`
`gen_ai.usage.output_tokens`	`2459`
`gen_ai.usage.output_tokens.reasoning`	`1931`
`gen_ai.usage.total_tokens`	`4548`
`thread.id`	`132539277981568`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-tool-call · response_id #
expected chatcmpl-wt-stools, reported nothing

Expected: chatcmpl-wt-stools
Reproduce: spantruth run --subject sentry --scenario openai/stream-tool-call
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.01293252699997538`
`gen_ai.system`	`openai`
`gen_ai.tool.definitions`	`[{"type": "function", "name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"type": "function", "name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}]`
`gen_ai.usage.input_tokens`	`1553`
`gen_ai.usage.output_tokens`	`173`
`gen_ai.usage.total_tokens`	`1726`
`thread.id`	`131551758366592`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/stream-tool-call · finish_reasonsSilent stop #
expected tool_calls, reported nothing

Expected: ['tool_calls']
Reproduce: spantruth run --subject sentry --scenario openai/stream-tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`True`
`gen_ai.response.time_to_first_token`	`0.01293252699997538`
`gen_ai.system`	`openai`
`gen_ai.tool.definitions`	`[{"type": "function", "name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"type": "function", "name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}]`
`gen_ai.usage.input_tokens`	`1553`
`gen_ai.usage.output_tokens`	`173`
`gen_ai.usage.total_tokens`	`1726`
`thread.id`	`131551758366592`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/tool-call · response_id #
expected chatcmpl-wt-tools, reported nothing

Expected: chatcmpl-wt-tools
Reproduce: spantruth run --subject sentry --scenario openai/tool-call
Rule: The unique identifier of the completion.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.tool.definitions`	`[{"type": "function", "name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"type": "function", "name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}]`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`gen_ai.usage.total_tokens`	`1019`
`thread.id`	`138852816652160`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

missing sentry · openai/tool-call · finish_reasonsSilent stop #
expected tool_calls, reported nothing

Expected: ['tool_calls']
Reproduce: spantruth run --subject sentry --scenario openai/tool-call
Rule: One finish reason per choice, as received. A normalized value is accepted too.

Exported spans (2)

chat gpt-5-mini sentry_sdk.adapter, status UNSET

`gen_ai.operation.name`	`chat`
`gen_ai.request.model`	`gpt-5-mini`
`gen_ai.response.model`	`gpt-5-mini-2026-08-07`
`gen_ai.response.streaming`	`False`
`gen_ai.system`	`openai`
`gen_ai.tool.definitions`	`[{"type": "function", "name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}, {"type": "function", "name": "get_time", "parameters": {"type": "object", "properties": {"zone": {"type": "string"}}}}]`
`gen_ai.usage.input_tokens`	`931`
`gen_ai.usage.output_tokens`	`88`
`gen_ai.usage.total_tokens`	`1019`
`thread.id`	`138852816652160`
`thread.name`	`MainThread`

spantruth.scenario spantruth.driver, status UNSET

Subjects and versions

anthropic-sdk: Anthropic SDK, built-in tracing. The SDK's own spans, with no instrumentation library. On by default since anthropic 1.14.0 when a tracer provider is registered. anthropic 2.4.0, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-sdk 1.45.2
ddtrace: Datadog ddtrace LLM Observability. ddtrace spans are not OpenTelemetry spans. A local adapter in instrument.py shows ddtrace's own LLM spans to the harness: tag and metric keys and values are unchanged, span.error becomes ERROR, and a span ddtrace never finishes stays open and is lost. span.parented comes from the adapter, not from ddtrace, whose spans form their own trace. What is graded is the APM span with its gen_ai.* tags, not the LLMObs event sent to Datadog, which holds finish_reason in its metadata. OpenAI's response model is also on the tag openai.response.model, a name the suite does not read. No Agent and no account: LLMObs.enable(agentless_enabled=False) tries a local Agent on 127.0.0.1:8126, finds none and drops the events. Telemetry and remote configuration are off. anthropic 2.4.0, ddtrace 4.16.3, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-sdk 1.45.2
langfuse: Langfuse Python SDK. The documented setup: from langfuse.openai import openai, with a throwaway key so Langfuse does not turn itself off. base_url points at the harness receiver so Langfuse's own export does not attempt an outbound connection; any POST to its own path (/api/public/otel/v1/traces) is answered 200 by the receiver. The driver's tracer provider exports the span to /v1/traces. anthropic 2.4.0, langfuse 5.2.0, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-sdk 1.45.2
mlflow: MLflow Tracing. MLflow's own OTLP export with its own option MLFLOW_ENABLE_OTEL_GENAI_SEMCONV=true, which translates mlflow.* attributes to GenAI names before export. No adapter. The findings on tokens and models are about that translation: MLflow's native attribute mlflow.chat.tokenUsage keeps the cache counts. The driver's provider also exports the untranslated copy of each span, which the suite does not grade. MLFLOW_USE_DEFAULT_TRACER_PROVIDER=false joins the app's trace, and mlflow.tracing.enable() sets up the processors before the app's parent span starts. MLFLOW_DISABLE_TELEMETRY=true stops the usage telemetry fetch and post (mlflow/telemetry/client.py). anthropic 2.4.0, mlflow-skinny 3.18.0, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-sdk 1.45.2
openinference: OpenInference. anthropic 2.4.0, openai 3.42.1, openinference-instrumentation-anthropic 3.3.0, openinference-instrumentation-openai 0.1.69, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-sdk 1.45.2
openlit: OpenLIT. anthropic 0.125.0, openai 2.54.0, openlit 1.45.0, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-sdk 1.45.2
openllmetry: OpenLLMetry. anthropic 2.4.0, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-instrumentation-anthropic 0.62.4, opentelemetry-instrumentation-openai 0.62.4, opentelemetry-sdk 1.45.2
otel-official: OpenTelemetry GenAI instrumentation. anthropic 2.4.0, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-instrumentation-genai-anthropic 1.3b0, opentelemetry-instrumentation-genai-openai 1.3b0, opentelemetry-sdk 1.45.2
otel-official-sdk-on: OpenTelemetry GenAI instrumentation, with the Anthropic SDK's own tracing left on. What an app gets by default: this library passes every scenario alone, so each difference here comes from the SDK tracing itself since anthropic 1.14.0. The other subjects turn the SDK's spans off with ANTHROPIC_OPEN_TELEMETRY=false. anthropic 2.4.0, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-instrumentation-genai-anthropic 1.3b0, opentelemetry-sdk 1.45.2
sentry: Sentry SDK. Sentry spans are not OpenTelemetry spans, and the SDK has no OTLP export for them (its OTLP integration sends OpenTelemetry spans to Sentry). A local adapter in instrument.py shows the SDK's own gen_ai spans to the harness: span data keys and values are unchanged, a Sentry status other than ok becomes ERROR, and a span the SDK never finishes stays open and is lost. span.parented comes from the adapter, not from Sentry, whose spans form their own trace. No DSN and no account: without a DSN the SDK has no transport, so nothing is sent. anthropic 2.4.0, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-sdk 1.45.2, sentry-sdk 2.74.0
control-faulty: Control: reference instrumentation with four planted faults. Must fail exactly the checks listed in expected.toml. If it does not, the suite is wrong. anthropic 2.4.0, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-sdk 1.45.2
control-honest: Control: reference instrumentation. Must pass every check. If it does not, the suite is wrong, not a library. anthropic 2.4.0, openai 3.42.1, opentelemetry-exporter-otlp-proto-http 1.45.2, opentelemetry-sdk 1.45.2