fix(llmobs): include missing tags and make SubmitEvaluationFromSpan more flexible by rarguelloF · Pull Request #4050 · DataDog/dd-trace-go

rarguelloF · 2025-10-15T15:43:08Z

What does this PR do?

Fix some minor issues in the llmobs SDK highlighted by the shared testing suite:

Fix the debug logs informing about whether agent / agentless mode is being used (they were displaying the opposite of what was actually being set). Additionally it returns an error when using incompatible setups (using agent mode and no agent is available).
Change the interface accepted for spans in SubmitEvaluationFromSpan to be less strict to support the use case of submitting evaluations when not having an actual span object (only span/trace IDs).
Include missing tag session_id in tags. It was currently only being included in the root field of the span event payload, but it should be present in both places.
Include the missing tag ddtrace.version in evaluation metrics.

Motivation

Correct/consistent implementation of the Go LLMObs SDK.

Reviewer's Checklist

Changed code has unit tests for its functionality at or near 100% coverage.
System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
There is a benchmark for any new code, or changes to existing code.
If this interacts with the agent in a new way, a system test has been added.
New code is free of linting errors. You can check this by running ./scripts/lint.sh locally.
Add an appropriate team label so this PR gets put in the right place for the release notes.
Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

… flexible

datadog-datadog-prod-us1 · 2025-10-15T15:59:39Z

⚠️ Tests

⚠️ Warnings

🧪 1 Test failed

TestMain from github.com/DataDog/dd-trace-go/v2/ddtrace/tracer (Datadog)

Failed

panic: test timed out after 10m0s
	running tests:
		TestSpanString (8m51s)

goroutine 4951 [running]:
testing.(*M).startAlarm.func1()
	/opt/hostedtoolcache/go/1.25.3/x64/src/testing/testing.go:2682 +0x605
created by time.goFunc
...

ℹ️ Info

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 27b29c6 | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-10-15T16:20:41Z

Benchmarks

Benchmark execution time: 2025-10-21 15:33:28

Comparing candidate commit 27b29c6 in PR branch rarguelloF/llmobs-fixes with baseline commit 45246a0 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 metrics, 0 unstable metrics.

sabrenner

nice lgtm! just a couple noob Go questions 😄

sabrenner · 2025-10-21T13:43:43Z

+	}
 	if cfg.AgentlessEnabled != nil {
+		if !*cfg.AgentlessEnabled && !agentSupportsLLMObs {
+			log.Warn("llmobs: DD_LLMOBS_AGENTLESS_ENABLED has been configured to false and the agent is not available or does not support llmobs")


should we return an error in this case? i don't think llmobs will be able to submit data in the scenario where there's no agent running and agentlessEnabled is explicitly set to false, and maybe we wanna surface that to the user harsher than a warning log, but not sure of the protocols for doing that in the go tracer!

hmm yes! this totally makes sense, I will change it 👍

…ode is explicitly setup

rarguelloF · 2025-10-21T15:23:57Z

/merge

dd-devflow-routing-codex · 2025-10-21T19:57:18Z

View all feedbacks in Devflow UI.

2025-10-21 15:43:51 UTC ℹ️ Start processing command /merge

2025-10-21 15:56:53 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2025-10-21 19:57:18 UTC ⚠️ MergeQueue: This merge request was unqueued

devflow unqueued this merge request: It did not become mergeable within the expected time

rarguelloF · 2025-10-22T08:23:23Z

/merge

dd-devflow-routing-codex · 2025-10-22T08:23:27Z

View all feedbacks in Devflow UI.

2025-10-22 08:23:26 UTC ℹ️ Start processing command /merge

2025-10-22 08:23:31 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 18m (p90).

2025-10-22 08:37:26 UTC ℹ️ MergeQueue: This merge request was merged

…ore flexible (#4050)

fix(llmobs): fix confusing logs and make eval metrics from spans more…

0a2650d

… flexible

fix(llmobs): set session_id tag

d0968e5

rarguelloF added 2 commits October 15, 2025 18:41

fix(llmobs): support all fields for EmbeddedDocument

722b866

fix(llmobs): include ddtrace.version tag in eval metrics

3b1c5e7

rarguelloF changed the title ~~fix(llmobs): fix confusing logs and make SubmitEvaluationFromSpan more flexible~~ fix(llmobs): include missing tags and make SubmitEvaluationFromSpan more flexible Oct 21, 2025

rarguelloF marked this pull request as ready for review October 21, 2025 07:53

rarguelloF requested a review from darccio October 21, 2025 07:56

darccio approved these changes Oct 21, 2025

View reviewed changes

rarguelloF requested a review from sabrenner October 21, 2025 08:14

sabrenner approved these changes Oct 21, 2025

View reviewed changes

feat(llmobs): return an error when agent is not supported and agent-m…

db6c72f

…ode is explicitly setup

rarguelloF requested review from a team as code owners October 21, 2025 14:35

fix test

27b29c6

rarguelloF requested a review from sabrenner October 21, 2025 15:20

sabrenner approved these changes Oct 21, 2025

View reviewed changes

dd-devflow Bot added mergequeue-status: waiting mergequeue-status: removed and removed mergequeue-status: waiting labels Oct 21, 2025

dd-devflow Bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: removed mergequeue-status: queued labels Oct 22, 2025

dd-mergequeue Bot merged commit be655e9 into main Oct 22, 2025
269 of 273 checks passed

dd-devflow Bot removed the mergequeue-status: in_progress label Oct 22, 2025

dd-mergequeue Bot deleted the rarguelloF/llmobs-fixes branch October 22, 2025 08:37

dd-devflow Bot added the mergequeue-status: done label Oct 22, 2025

genesor pushed a commit that referenced this pull request Oct 30, 2025

fix(llmobs): include missing tags and make SubmitEvaluationFromSpan m…

a733fee

…ore flexible (#4050)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(llmobs): include missing tags and make SubmitEvaluationFromSpan more flexible#4050

fix(llmobs): include missing tags and make SubmitEvaluationFromSpan more flexible#4050
dd-mergequeue[bot] merged 6 commits into
mainfrom
rarguelloF/llmobs-fixes

rarguelloF commented Oct 15, 2025 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 Bot commented Oct 15, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Oct 15, 2025 •

edited

Loading

Uh oh!

sabrenner left a comment

Uh oh!

sabrenner Oct 21, 2025

Uh oh!

rarguelloF Oct 21, 2025

Uh oh!

Uh oh!

rarguelloF commented Oct 21, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Oct 21, 2025 •

edited

Loading

Uh oh!

rarguelloF commented Oct 22, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Oct 22, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

rarguelloF commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Reviewer's Checklist

Uh oh!

datadog-datadog-prod-us1 Bot commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

pr-commenter Bot commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Uh oh!

sabrenner left a comment

Choose a reason for hiding this comment

Uh oh!

sabrenner Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

rarguelloF Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rarguelloF commented Oct 21, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rarguelloF commented Oct 22, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rarguelloF commented Oct 15, 2025 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Oct 15, 2025 •

edited

Loading

pr-commenter Bot commented Oct 15, 2025 •

edited

Loading

dd-devflow-routing-codex Bot commented Oct 21, 2025 •

edited

Loading

dd-devflow-routing-codex Bot commented Oct 22, 2025 •

edited

Loading