Skip to content

fix(llmobs): do not cancel the context before reading response bodies#4075

Merged
dd-mergequeue[bot] merged 4 commits into
mainfrom
rarguelloF/improve-llmobs-errors
Oct 28, 2025
Merged

fix(llmobs): do not cancel the context before reading response bodies#4075
dd-mergequeue[bot] merged 4 commits into
mainfrom
rarguelloF/improve-llmobs-errors

Conversation

@rarguelloF

@rarguelloF rarguelloF commented Oct 27, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

What does this PR do?

We were reading the response body after the request context is canceled, which was triggering an error in the io.ReadAll(resp.Body) call in certain cases (observed it happening when the response included the header Transfer-Encoding: chunked).

Additionally, it improves the error messages for better debugging and checks for the presence of an app key in dataset.Pull when running in agentless mode, similar as the other dataset public functions.

Motivation

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • New code is free of linting errors. You can check this by running ./scripts/lint.sh locally.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

@pr-commenter

pr-commenter Bot commented Oct 27, 2025 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2025-10-28 15:27:11

Comparing candidate commit e0c6721 in PR branch rarguelloF/improve-llmobs-errors with baseline commit 99623ba in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 metrics, 0 unstable metrics.

@rarguelloF rarguelloF changed the title chore(llmobs): include server response in llmobs errors fix(llmobs): do not cancel the context before reading response bodies Oct 27, 2025
@rarguelloF
rarguelloF marked this pull request as ready for review October 27, 2025 21:02
@rarguelloF
rarguelloF requested a review from a team as a code owner October 27, 2025 21:02
gary-huang
gary-huang approved these changes Oct 27, 2025
View reviewed changes

@gary-huang gary-huang left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a few comments about log lines

thanks!

Comment thread internal/llmobs/transport/dne.go
Comment thread internal/llmobs/transport/dne.go
Comment thread internal/llmobs/transport/transport.go Outdated
if !strings.Contains(contentType, "application/json") {
return ""
}
// Limit reading to 1KB to avoid reading huge error responses

@gary-huang gary-huang Oct 27, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess it's fine to start like this but why not read the whole thing? imo this is internal use (for now) so there is no issue with reading big error responses, if anything it would be helpful for debugging?

@rarguelloF rarguelloF Oct 28, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense! will change

@rarguelloF

rarguelloF commented Oct 28, 2025

Copy link
Copy Markdown
Contributor Author

/merge

@dd-devflow-routing-codex

dd-devflow-routing-codex Bot commented Oct 28, 2025 •
edited
Loading

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2025-10-28 17:53:40 UTC ℹ️ Start processing command /merge

2025-10-28 17:53:45 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 18m (p90).

2025-10-28 18:18:09 UTC ℹ️ MergeQueue: This merge request was merged

@dd-mergequeue
dd-mergequeue Bot merged commit 3b824da into main Oct 28, 2025
243 checks passed
@dd-mergequeue
dd-mergequeue Bot deleted the rarguelloF/improve-llmobs-errors branch October 28, 2025 18:18
genesor pushed a commit that referenced this pull request Oct 30, 2025
@rarguelloF @genesor
fix(llmobs): do not cancel the context before reading response bodies (…
e72e553 
…#4075)

Co-authored-by: rodrigo.arguello <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@darccio darccio darccio approved these changes
+1 more reviewer
@gary-huang gary-huang gary-huang approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

mergequeue-status: done

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rarguelloF @darccio @gary-huang