Skip to content

fix(tracing): record zero top_k / chunk_size in update_retriever_span - #3406

Open
Lesereingrape wants to merge 1 commit into
confident-ai:mainfrom
Lesereingrape:fix/retriever-span-zero-values-1003
Open

Lesereingrape wants to merge 1 commit into
confident-ai:mainfrom
Lesereingrape:fix/retriever-span-zero-values-1003

Conversation

@Lesereingrape

@Lesereingrape Lesereingrape commented Oct 3, 2026 •

Copy link
Copy Markdown

What

update_retriever_span(...) guarded top_k and chunk_size on truthiness, so a caller that legitimately reports top_k=0 (or chunk_size=0) silently gets the value dropped instead of recorded on the span.

This changes both guards to is not None, which is what the signature already implies: None is the "not provided" marker, so 0 should be a recordable value.

Why

next_retriever_span(...) already persists zeros for these fields through _drop_none, so the two entry points disagree — the same trace built through the update path loses data that the constructor path keeps.

How I checked

Added tests/test_core/test_tracing/test_update_functions/test_update_retriever_span_zero_values.py:

  • test_zero_top_k_and_chunk_size_are_recorded — fails on current main with AssertionError: assert None == 0, passes with the guard change.
  • test_omitted_fields_are_left_untouched — pins the existing behavior that omitted (None) fields must not overwrite values already on the span.

Measured on origin/main (a200ecede) with the fix reverted vs applied:

# reverted, only the new file
1 failed, 1 passed   (test_zero_top_k_and_chunk_size_are_recorded -> assert None == 0)
# applied
tests/test_core/test_tracing/test_update_functions/ -> 18 passed in 2.25s

embedder keeps its truthiness guard on purpose — an empty embedder name carries no information, unlike a zero count.

Notes
  • No linked issue; found while reading deepeval/tracing/context.py.
  • Contributed with AI assistance (a coding agent drafted the patch and ran the tests above); the reproduction, the measured RED/GREEN results and this description are from the actual diff and test output.

CI note

This branch has 11/13 checks green; the two red test shards fail on tests that are already red on main (a200ecede) with the same node id and the same message:

  • tests/test_core/test_sqlite_store.py::TestTestRunManagerIntegration::test_storage_failure_is_a_warning_not_an_error — AssertionError: assert 'disk I/O error' in 'Warning: fa…' (the rich 80-column wrap of the saved-run path in deepeval/test_run/test_run.py).
  • tests/test_metrics/test_eval_mode_metrics.py::test_hybrid_jev_takes_the_decisions[sync-MultiTurnMCPUse] — AttributeError: 'CallToolResult' object has no attribute 'structuredContent'.

Both look unrelated to this diff, and #3316 / #3373 appear to already target them. Because the core shard runs with --maxfail=1, it stops at the sqlite failure before reaching tests/test_core/test_tracing/, so the tests added here don't show up in that log; locally this whole directory is green (18 passed).

`None` is the "not provided" marker in this signature, so truthiness
guards swallow a legitimately reported 0 and leave the span field
untouched. `update_llm_span` already documents this exact rule for its
numeric fields, and `next_retriever_span` keeps 0 on its way through
`_drop_none`, so the declarative and imperative paths disagreed.
@vercel

vercel Bot commented Oct 3, 2026

Copy link
Copy Markdown

@Lesereingrape is attempting to deploy a commit to the Confident AI Team on Vercel.

A member of the Team first needs to authorize it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant