fix: record errors on dropped llama_index spans - #3305

Summary

LLamaIndexHandler.prepare_to_drop_span ignored its err argument, so a LlamaIndex span that ended by raising was finalized exactly like one that completed: the span was stamped SUCCESS with no error message, its root trace ended as SUCCESS, and the span was never removed from trace_manager.active_spans.

Why

llama_index_instrumentation only calls span_drop(..., err=e) from except BaseException blocks (llama_index_instrumentation/dispatcher.py, lines 397, 435 and 467), so every span that reaches prepare_to_drop_span is a failed span.

In deepeval/integrations/llama_index/handler.py before this change:

• line 399 set base_span.status = TraceSpanStatus.SUCCESS unconditionally, discarding err.
• line 402 called trace_manager.end_trace(...) for root spans, and end_trace defaults an IN_PROGRESS trace to SUCCESS (deepeval/tracing/tracing.py:352-353) — so a failed workflow or agent run was reported as a successful trace with no error.
• the function never called trace_manager.remove_span(...), unlike its sibling prepare_to_exit_span (same file, line 412), so every errored span stayed in active_spans for the lifetime of the process.

The equivalent path is already handled correctly for LangChain in deepeval/integrations/langchain/utils.py:446-449 and :484-485: set ERRORED plus error = str(exc_val), mark the trace errored, then remove_span. This change brings prepare_to_drop_span in line with that.

Changes

deepeval/integrations/llama_index/handler.py, only prepare_to_drop_span:

• err is not None sets base_span.status = TraceSpanStatus.ERRORED and base_span.error = str(err); otherwise the status stays SUCCESS.
• for a root span, the trace is set to ERRORED before end_trace is called, so end_trace cannot flip it back to SUCCESS.
• trace_manager.remove_span(base_span.uuid) is called at the end, matching prepare_to_exit_span.

The else-branch (SUCCESS) behavior for a drop with no exception is unchanged.

Testing

New regression test: tests/test_integrations/test_llamaindex/test_drop_span.py. Four tests drive prepare_to_drop_span against a synthetic span registered in trace_manager.active_spans (no LLM or API key needed), plus one end-to-end test that decorates a raising function with the real LlamaIndex dispatcher and asserts the posted trace comes back ERRORED with the error message and that active_spans is empty.

On the unpatched tree:

$ python -m pytest tests/test_integrations/test_llamaindex/test_drop_span.py -q
FFF.F                                                                    [100%]
...
E       AssertionError: assert <TraceSpanStatus.SUCCESS: 'SUCCESS'> == <TraceSpanStatus.ERRORED: 'ERRORED'>
...
FAILED tests/test_integrations/test_llamaindex/test_drop_span.py::test_dropped_span_records_error
FAILED tests/test_integrations/test_llamaindex/test_drop_span.py::test_dropped_span_is_removed_from_active_spans
FAILED tests/test_integrations/test_llamaindex/test_drop_span.py::test_dropped_root_span_marks_trace_errored
FAILED tests/test_integrations/test_llamaindex/test_drop_span.py::test_instrumented_failure_ends_errored_trace
4 failed, 1 passed in 0.12s

With the fix:

$ python -m pytest tests/test_integrations/test_llamaindex/test_drop_span.py -q
5 passed in 0.06s

Existing suites:

$ python -m pytest tests/test_core/test_tracing -q
217 passed, 1 skipped in 5.87s

$ python -m pytest tests/test_integrations/test_llamaindex/ -q
10 failed, 5 passed in 0.34s

The 10 failures in test_sync.py / test_async.py are pre-existing in this environment and unrelated to this change — they all raise ValueError: No API key found for OpenAI. while constructing the test apps' embeddings, and the same 10 fail on unmodified main.

Lint (the repo's CI gate is psf/black --check, pinned 25.12.0):

$ black --check deepeval/integrations/llama_index/handler.py tests/test_integrations/test_llamaindex/test_drop_span.py
All done! 2 files would be left unchanged.

ruff check reports one issue, E722 Do not use bare except at handler.py:60, which is pre-existing and byte-identical on main; that line is not touched by this diff.