feat(llmobs): add LLM Observability SDK by rarguelloF · Pull Request #3938 · DataDog/dd-trace-go

rarguelloF · 2025-09-02T12:42:51Z

What does this PR do?

This PR adds the Go SDK for the DataDog LLM Observability product (https://docs.datadoghq.com/llm_observability/instrumentation/sdk/).

It includes support for the main LLM Observability features:

Creating/annotating spans.
Context propagation.
Submitting evaluation metrics.
Datasets and Experiments.

It includes the following new options and changes in the ddtrace/tracer package:

WithLLMObsEnabled (equivalent to DD_LLMOBS_ENABLED) - false by default.
WithLLMObsMLApp (equivalent to DD_LLMOBS_ML_APP)
WithLLMObsProjectName (equivalent to DD_LLMOBS_PROJECT_NAME)
WithLLMObsAgentlessEnabled (equivalent to DD_LLMOBS_AGENTLESS_ENABLED)
Now the functions tracer.Start|Stop|Flush call internal/llmobs.Start|Stop|Flush respectively if llmobs is enabled and forwards the error (if applicable).
Raises the default max size of the X-Datadog-Tags header from 128 to 512. This was necessary to include the llmobs headers required for context propagation, and aligns with the default value in other tracers.

The llmobs public api is divided in the following packages:

llmobs contains the main spans / metrics functionality.
llmobs/experiment contains the api to work with LLM Observability Experiments.
llmobs/dataset contains the api to work with LLM Observability Dataset.

The llmobs internal api is divided in the following packages:

internal/llmobs: contains most of the core logic of llmobs, including the main worker that receives spans/metrics and flushes periodically or by demand.
internal/llmobs/transport: contains all the payload definitions and logic to communicate with the LLMObs backend.
internal/llmobs/config: contains the global configuration for all LLMObs features.

Motivation

Support DataDog LLM Observability in dd-trace-go.

Reviewer's Checklist

Changed code has unit tests for its functionality at or near 100% coverage.
System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
There is a benchmark for any new code, or changes to existing code.
If this interacts with the agent in a new way, a system test has been added.
New code is free of linting errors. You can check this by running ./scripts/lint.sh locally.
Add an appropriate team label so this PR gets put in the right place for the release notes.
Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

pr-commenter · 2025-09-02T12:46:01Z

Benchmarks

Benchmark execution time: 2025-10-08 14:15:56

Comparing candidate commit 3c58818 in PR branch rarguelloF/AIDM-708/llmobs-client with baseline commit 23545d4 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 15 metrics, 0 unstable metrics.

datadog-datadog-prod-us1 · 2025-09-02T12:51:58Z

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 3c58818 | Docs | Was this helpful? Give us feedback!}

hannahkm · 2025-09-04T15:47:41Z

+// This product includes software developed at Datadog (https://www.datadoghq.com/).
+// Copyright 2025 Datadog, Inc.
+
+package errors


We have a similar package that we merged a bit ago. Could it cover the use cases we have here, instead of creating a new package for it? We can add more functionality to the other package as needed.

yes! I didn't know this existed. I already refactored the code to use this package, will push the changes shortly 🙏

rarguelloF · 2025-10-06T10:44:15Z

feat(llmobs): add LLM Observability SDK #3938 : 2 dependent PRs (#4024 , #4032 ) 👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

darccio · 2025-10-06T15:15:02Z

+// ------------- Start options -------------
+
+// StartSpanOption configures span creation. Use with Start*Span functions.
+type StartSpanOption = func(cfg *illmobs.StartSpanConfig)


@rarguelloF Is there any reason to use an alias here instead of defining the type as it's done here?

nice catch! this was unintended, will change it.

darccio · 2025-10-06T15:15:23Z

+// ------------- Finish options -------------
+
+// FinishSpanOption configures span finishing. Use with span.Finish().
+type FinishSpanOption = func(cfg *illmobs.FinishSpanConfig)


Previous comment applies here too.

darccio · 2025-10-06T17:25:41Z

It's possible I didn't catch more cases where I prefer to invert the conditional and exit earlier. I'm applying "The happy path is left-aligned" idiom.

darccio · 2025-10-06T17:33:15Z

@rarguelloF Please check my PR for avoiding interfaces as returned values #4028

…evaluators (#4025)

genesor

LGTM, small comments about comments that we could benefit from.

genesor · 2025-10-01T13:24:54Z

+	"github.com/DataDog/dd-trace-go/v2/internal/llmobs"
+)
+
+type llmobsTracerAdapter struct{}


A comment here explaining why it exists, how and when it's used would be great

makes sense! adding it

genesor · 2025-10-01T13:25:06Z

+	return &llmobsSpanAdapter{span}, ctx
+}
+
+type llmobsSpanAdapter struct {


genesor · 2025-10-08T12:40:59Z

+	return illmobs.ContextWithPropagatedLLMSpan(newCtx, propagatedLLMObs)
+}
+
+func propagatedLLMSpanFromTags(s *Span) *illmobs.PropagatedLLMSpan {


shouldn't this function be in the llmobs package ?

this function reads the span's trace propagating tags (which are basically the tags that come from the X-Datadog-Tags header), which is all in private fields, so it cannot be outside of this package. I will add a clarifying comment on this function 👍

rarguelloF · 2025-10-08T14:01:07Z

/merge

dd-devflow-routing-codex · 2025-10-08T14:01:11Z

View all feedbacks in Devflow UI.

2025-10-08 14:01:10 UTC ℹ️ Start processing command /merge

2025-10-08 14:01:21 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2025-10-08 14:34:38 UTC ℹ️ MergeQueue: merge request added to the queue

The expected merge time in main is approximately 18m (p90).

2025-10-08 14:53:54 UTC ℹ️ MergeQueue: This merge request was merged

Co-authored-by: darccio <[email protected]>

feat(llmobs): add LLM Observability SDK

890e929

github-advanced-security AI found potential problems Sep 2, 2025

View reviewed changes

Comment thread llmobs/internal/dne_client.go Fixed

hannahkm reviewed Sep 4, 2025

View reviewed changes

rarguelloF added 14 commits September 5, 2025 14:15

WIP: fixes on dne

9c493ee

fix path for unix data socket

00310d9

wip: llmobs spans

3497dff

fix llmobs span payload

e7409cf

integrate llmobs in the tracer

0fa7de7

distributed context propagation + tests

79bd066

fix context propagation

1cfadb9

annotation api

baddd39

fixes

4b62c89

add RequestDelay option

309d9f7

performance improvements

312deff

refactor payload names

5e1c1aa

Merge branch 'main' into rarguelloF/AIDM-708/llmobs-client

d9027c0

update go.mod

98f13b2

github-advanced-security AI found potential problems Sep 24, 2025

View reviewed changes

Comment thread internal/llmobs/transport/transport.go Fixed

rarguelloF added 10 commits September 24, 2025 17:24

add tests

d20ed62

update configurations

8703018

tests and bug fixes

2524cc8

add godoc

7bb674b

add eval metrics

992fe96

fix metrics and nil handling

2004aa1

add tests

e08f61f

add go docs

d44e46c

fix tests

0932552

fix lint

ef1ed27

rarguelloF added 2 commits October 1, 2025 19:19

fix tests

dbf0af4

fix tests

5e06085

This was referenced Oct 6, 2025

fix(llmobs/dataset): input and metadata can be any type #4024

Merged

refactor(llmobs/experiment): provide the entire record for tasks and evaluators #4025

Merged

darccio requested changes Oct 6, 2025

View reviewed changes

feat(llmobs): refactor public API to use concrete types (#4028)

fa28859

darccio approved these changes Oct 7, 2025

View reviewed changes

rarguelloF added 5 commits October 7, 2025 17:22

fix(llmobs/dataset): input and metadata can be any type (#4024)

bfdf029

refactor(llmobs/experiment): provide the entire record for tasks and …

9ceca95

…evaluators (#4025)

address review comments

19d568d

Merge branch 'main' into rarguelloF/AIDM-708/llmobs-client

79a3f46

clean up api

824dc80

rarguelloF mentioned this pull request Oct 7, 2025

fix(llmobs/dataset): add pagination and big datasets support #4032

Closed

7 tasks

genesor approved these changes Oct 8, 2025

View reviewed changes

address comments

3c58818

dd-devflow Bot added mergequeue-status: waiting mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: waiting mergequeue-status: queued labels Oct 8, 2025

dd-mergequeue Bot merged commit 5603e37 into main Oct 8, 2025
352 of 355 checks passed

dd-devflow Bot added mergequeue-status: done and removed mergequeue-status: in_progress labels Oct 8, 2025

dd-mergequeue Bot deleted the rarguelloF/AIDM-708/llmobs-client branch October 8, 2025 14:53

hannahkm pushed a commit that referenced this pull request Oct 9, 2025

feat(llmobs): add LLM Observability SDK (#3938)

51ba83b

Co-authored-by: darccio <[email protected]>

Uh oh!

Conversation

rarguelloF commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Reviewer's Checklist

Uh oh!

pr-commenter Bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Uh oh!

Uh oh!

datadog-datadog-prod-us1 Bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rarguelloF commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

darccio commented Oct 6, 2025

Uh oh!

darccio commented Oct 6, 2025

Uh oh!

genesor left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rarguelloF commented Oct 8, 2025

Uh oh!

dd-devflow-routing-codex Bot commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

rarguelloF commented Sep 2, 2025 •

edited

Loading

pr-commenter Bot commented Sep 2, 2025 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Sep 2, 2025 •

edited

Loading

rarguelloF commented Oct 6, 2025 •

edited

Loading

dd-devflow-routing-codex Bot commented Oct 8, 2025 •

edited

Loading