feat: single source of truth for headers (fixes issue in profiling with missing headers)#1493
Conversation
📚 Documentation Check Results📦
|
🔒 Cargo Deny Results📦
|
BenchmarksComparisonBenchmark execution time: 2026-02-04 18:53:04 Comparing candidate commit 7ac277e in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 2 unstable metrics. CandidateCandidate benchmark detailsGroup 1
Group 2
Group 3
Group 4
Group 5
Group 6
Group 7
Group 8
Group 9
Group 10
Group 11
Group 12
Group 13
Group 14
Group 15
Group 16
Group 17
Group 18
Group 19
BaselineOmitted due to size. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1493 +/- ##
==========================================
+ Coverage 71.04% 71.25% +0.20%
==========================================
Files 424 424
Lines 69307 69781 +474
==========================================
+ Hits 49240 49722 +482
+ Misses 20067 20059 -8
🚀 New features to boost your workflow:
|
ivoanjo
left a comment
There was a problem hiding this comment.
👍 LGTM, Added a note about testing
| pub fn assert_entity_headers_match(headers: &HashMap<String, String>) { | ||
| // Check for entity headers and validate their values match what libdd_common provides | ||
| let expected_container_id = libdd_common::entity_id::get_container_id(); | ||
| let expected_entity_id = libdd_common::entity_id::get_entity_id(); | ||
| let expected_external_env = *libdd_common::entity_id::DD_EXTERNAL_ENV; | ||
|
|
||
| // Validate container ID | ||
| if let Some(expected) = expected_container_id { | ||
| assert_eq!( | ||
| headers.get("datadog-container-id"), | ||
| Some(&expected.to_string()), | ||
| "datadog-container-id header should match the value from entity_id::get_container_id()" | ||
| ); | ||
| } else { | ||
| assert!( | ||
| !headers.contains_key("datadog-container-id"), | ||
| "datadog-container-id header should not be present when entity_id::get_container_id() is None" | ||
| ); | ||
| } | ||
|
|
||
| // Validate entity ID | ||
| if let Some(expected) = expected_entity_id { | ||
| assert_eq!( | ||
| headers.get("datadog-entity-id"), | ||
| Some(&expected.to_string()), | ||
| "datadog-entity-id header should match the value from entity_id::get_entity_id()" | ||
| ); | ||
| } else { | ||
| assert!( | ||
| !headers.contains_key("datadog-entity-id"), | ||
| "datadog-entity-id header should not be present when entity_id::get_entity_id() is None" | ||
| ); | ||
| } | ||
|
|
||
| // Validate external env | ||
| if let Some(expected) = expected_external_env { | ||
| assert_eq!( | ||
| headers.get("datadog-external-env"), | ||
| Some(&expected.to_string()), | ||
| "datadog-external-env header should match the value from entity_id::DD_EXTERNAL_ENV" | ||
| ); | ||
| } else { | ||
| assert!( | ||
| !headers.contains_key("datadog-external-env"), | ||
| "datadog-external-env header should not be present when entity_id::DD_EXTERNAL_ENV is None" | ||
| ); | ||
| } |
There was a problem hiding this comment.
Hmmm I'm not the biggest fan of these tests, since they change their behavior dynamically based on the exact execution environment of the test runner, which means that
a) Whatever I get running on my machine may be different -- so maybe it fails in CI but never for me
b) We don't get full test coverage -- only whatever codepaths end up being picked
Is it possible to maybe mock the results of those functions for our tests?
That is, what I would do if this were Ruby would be to zoom out and say "the behavior in my headers is -- if each of these entries is available, the headers contain what got returned, and if there was nothing than the headers don't exist"
Then I'd mock container-id: [dummy-container-id, none], entity-id: [dummy-entity-id, none], external-env: [dummy-external-env, none] and check that I get the correct behavior in each of the 6 cases.
There was a problem hiding this comment.
Rust doesn't have great capabilities for mocking. I'm going to merge this, then post a followup PR with the mocking code since its somewhat intrusive and we can discuss the benefits there.
There was a problem hiding this comment.
It looks a bit like the underlying code, e.g. get_entity_id and such were not written in such a way to be testable. Maybe we should re-examine that angle in the follow-up PR, rather than mocking specifically? It looks like get_entity_id specifically might not be too hard based on looking at the source code.
There was a problem hiding this comment.
Agree that improving the test separately isn't a blocker. I would in any case leave a big comment above assert_entity_headers_match explaining the situation until we fix it?
| pub fn assert_entity_headers_match(headers: &HashMap<String, String>) { | ||
| // Check for entity headers and validate their values match what libdd_common provides | ||
| let expected_container_id = libdd_common::entity_id::get_container_id(); | ||
| let expected_entity_id = libdd_common::entity_id::get_entity_id(); | ||
| let expected_external_env = *libdd_common::entity_id::DD_EXTERNAL_ENV; | ||
|
|
||
| // Validate container ID | ||
| if let Some(expected) = expected_container_id { | ||
| assert_eq!( | ||
| headers.get("datadog-container-id"), | ||
| Some(&expected.to_string()), | ||
| "datadog-container-id header should match the value from entity_id::get_container_id()" | ||
| ); | ||
| } else { | ||
| assert!( | ||
| !headers.contains_key("datadog-container-id"), | ||
| "datadog-container-id header should not be present when entity_id::get_container_id() is None" | ||
| ); | ||
| } | ||
|
|
||
| // Validate entity ID | ||
| if let Some(expected) = expected_entity_id { | ||
| assert_eq!( | ||
| headers.get("datadog-entity-id"), | ||
| Some(&expected.to_string()), | ||
| "datadog-entity-id header should match the value from entity_id::get_entity_id()" | ||
| ); | ||
| } else { | ||
| assert!( | ||
| !headers.contains_key("datadog-entity-id"), | ||
| "datadog-entity-id header should not be present when entity_id::get_entity_id() is None" | ||
| ); | ||
| } | ||
|
|
||
| // Validate external env | ||
| if let Some(expected) = expected_external_env { | ||
| assert_eq!( | ||
| headers.get("datadog-external-env"), | ||
| Some(&expected.to_string()), | ||
| "datadog-external-env header should match the value from entity_id::DD_EXTERNAL_ENV" | ||
| ); | ||
| } else { | ||
| assert!( | ||
| !headers.contains_key("datadog-external-env"), | ||
| "datadog-external-env header should not be present when entity_id::DD_EXTERNAL_ENV is None" | ||
| ); | ||
| } |
There was a problem hiding this comment.
It looks a bit like the underlying code, e.g. get_entity_id and such were not written in such a way to be testable. Maybe we should re-examine that angle in the follow-up PR, rather than mocking specifically? It looks like get_entity_id specifically might not be too hard based on looking at the source code.
Co-authored-by: Levi Morrison <[email protected]>
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-apple-darwin
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-apple-darwin
x86_64-unknown-linux-gnu
|
bd81d7d to
b58d459
Compare
6d72d93 to
7ac277e
Compare
|
/merge |
|
View all feedbacks in Devflow UI.
This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
The expected merge time in
|
# Release proposal for libdd-common and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-common **Next version:** `1.2.0` **Semver bump:** `minor` **Tag:** `libdd-common-v1.2.0` ### Commits - feat: single source of truth for headers (fixes issue in profiling with missing headers) (#1493) - refactor!: make reqwest available in common (#1504) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
# Release proposal for libdd-common and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-common **Next version:** `1.2.0` **Semver bump:** `minor` **Tag:** `libdd-common-v1.2.0` ### Commits - feat: single source of truth for headers (fixes issue in profiling with missing headers) (#1493) - refactor!: make reqwest available in common (#1504) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
# Release proposal for libdd-common and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-common **Next version:** `1.2.0` **Semver bump:** `minor` **Tag:** `libdd-common-v1.2.0` ### Commits - feat: single source of truth for headers (fixes issue in profiling with missing headers) (#1493) - refactor!: make reqwest available in common (#1504) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
# Release proposal for libdd-data-pipeline and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-common **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-common-v2.0.0` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - feat!: enable non-blocking DNS for reqwest (#1558) - feat: unify Azure tags (#1553) - feat(common): add current thread id API (#1569) - refactor!: switch from multipart to multer to resolve deprecation warnings and dependabot alerts (#1540) - feat: single source of truth for headers (fixes issue in profiling with missing headers) (#1493) - refactor!: make reqwest available in common (#1504) ## libdd-ddsketch **Next version:** `1.0.1` **Semver bump:** `patch` **Tag:** `libdd-ddsketch-v1.0.1` ### Commits - chore: remove manual changelog modifications (#1472) - build: update `prost` crates (#1426) - chore: add changelog for every published crate (#1396) ## libdd-trace-protobuf **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-trace-protobuf-v1.1.0` ### Commits - chore: remove manual changelog modifications (#1472) - build: update `prost` crates (#1426) - chore: add changelog for every published crate (#1396) - Handle null span tag values (#1394) ## libdd-dogstatsd-client **Next version:** `1.0.1` **Semver bump:** `patch` **Tag:** `libdd-dogstatsd-client-v1.0.1` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - chore: release libddcommon-v1.1.0 (#1456) - chore: add changelog for every published crate (#1396) - style: fix recent clippy warnings (#1346) ## libdd-telemetry **Next version:** `3.0.0` **Semver bump:** `major` **Tag:** `libdd-telemetry-v3.0.0` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - chore(deps): bump tracing-subscriber to remove regex dep duplicate (#1608) - feat(telemetry)!: add process_tags to Application in telemetry (#1459) - fix(telemetry)!: fix logs payload format [APMSP-2590] (#1498) - feat(appsec): add endpoints collection (#1182) ## libdd-trace-normalization **Next version:** `1.0.1` **Semver bump:** `patch` **Tag:** `libdd-trace-normalization-v1.0.1` ### Commits - Prevent span start overflow panic (#1373) - [CHAOSPLT-932] Add support for internal fuzzing infra (#1372) - chore: add changelog for every published crate (#1396) ## libdd-trace-utils **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-utils-v2.0.0` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - feat: unify Azure tags (#1553) - fix(serverless): set hostname on stats from tracer to empty string (#1530) - chore: remove manual changelog modifications (#1472) - feat(sidecar)!: introduce TraceData to unify text and binary data (#1247) - chore: release libddcommon-v1.1.0 (#1456) - fix(test-agent): undo commenting arg in docker cmd (#1439) - [SLES-2652] Log error details when trace request fails (2) (#1441) - build: update `prost` crates (#1426) - chore(trace-utils): bump the test agent version used for integration tests (#1417) - [Serverless] Skip AAS metadata tagging when span is from API Management (#1409) - chore: add changelog for every published crate (#1396) - Handle null span tag values (#1394) - [SVLS-7934] Log error details when trace request fails (#1392) - Fix trace utils clippy warning (#1397) - feat(trace_utils): Allow sending trace stats using custom HTTP client (#1345) ## libdd-trace-stats **Next version:** `1.0.1` **Semver bump:** `patch` **Tag:** `libdd-trace-stats-v1.0.1` ### Commits - feat(sidecar)!: introduce TraceData to unify text and binary data (#1247) - chore: add changelog for every published crate (#1396) ## libdd-data-pipeline **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-v2.0.0` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - refactor(trace_exporter)!: remove Proxy TraceExporter input mode (#1583) - refactor(libdd-data-pipeline): health metrics (#1433) - feat(data-pipeline)!: include reason for chunks dropped telemetry (#1449) - feat(sidecar)!: introduce TraceData to unify text and binary data (#1247) - chore: release libddcommon-v1.1.0 (#1456) - chore: prepare libdd-telemetry-v2.0.0 (#1457) - Allow submitting Vec<Vec<Span>> asynchronously (#1302) - test(data-pipeline): handle EINTR in test_health_metrics_disabled (#1430) - chore: add changelog for every published crate (#1396) [APMSP-2590]: https://datadoghq.atlassian.net/browse/APMSP-2590?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>
# Release proposal for libdd-data-pipeline and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-common **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-common-v2.0.0` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - feat!: enable non-blocking DNS for reqwest (#1558) - feat: unify Azure tags (#1553) - feat(common): add current thread id API (#1569) - refactor!: switch from multipart to multer to resolve deprecation warnings and dependabot alerts (#1540) - feat: single source of truth for headers (fixes issue in profiling with missing headers) (#1493) - refactor!: make reqwest available in common (#1504) ## libdd-ddsketch **Next version:** `1.0.1` **Semver bump:** `patch` **Tag:** `libdd-ddsketch-v1.0.1` ### Commits - chore: remove manual changelog modifications (#1472) - build: update `prost` crates (#1426) - chore: add changelog for every published crate (#1396) ## libdd-trace-protobuf **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-trace-protobuf-v1.1.0` ### Commits - chore: remove manual changelog modifications (#1472) - build: update `prost` crates (#1426) - chore: add changelog for every published crate (#1396) - Handle null span tag values (#1394) ## libdd-dogstatsd-client **Next version:** `1.0.1` **Semver bump:** `patch` **Tag:** `libdd-dogstatsd-client-v1.0.1` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - chore: release libddcommon-v1.1.0 (#1456) - chore: add changelog for every published crate (#1396) - style: fix recent clippy warnings (#1346) ## libdd-telemetry **Next version:** `3.0.0` **Semver bump:** `major` **Tag:** `libdd-telemetry-v3.0.0` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - chore(deps): bump tracing-subscriber to remove regex dep duplicate (#1608) - feat(telemetry)!: add process_tags to Application in telemetry (#1459) - fix(telemetry)!: fix logs payload format [APMSP-2590] (#1498) - feat(appsec): add endpoints collection (#1182) ## libdd-trace-normalization **Next version:** `1.0.1` **Semver bump:** `patch` **Tag:** `libdd-trace-normalization-v1.0.1` ### Commits - Prevent span start overflow panic (#1373) - [CHAOSPLT-932] Add support for internal fuzzing infra (#1372) - chore: add changelog for every published crate (#1396) ## libdd-trace-utils **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-utils-v2.0.0` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - feat: unify Azure tags (#1553) - fix(serverless): set hostname on stats from tracer to empty string (#1530) - chore: remove manual changelog modifications (#1472) - feat(sidecar)!: introduce TraceData to unify text and binary data (#1247) - chore: release libddcommon-v1.1.0 (#1456) - fix(test-agent): undo commenting arg in docker cmd (#1439) - [SLES-2652] Log error details when trace request fails (2) (#1441) - build: update `prost` crates (#1426) - chore(trace-utils): bump the test agent version used for integration tests (#1417) - [Serverless] Skip AAS metadata tagging when span is from API Management (#1409) - chore: add changelog for every published crate (#1396) - Handle null span tag values (#1394) - [SVLS-7934] Log error details when trace request fails (#1392) - Fix trace utils clippy warning (#1397) - feat(trace_utils): Allow sending trace stats using custom HTTP client (#1345) ## libdd-trace-stats **Next version:** `1.0.1` **Semver bump:** `patch` **Tag:** `libdd-trace-stats-v1.0.1` ### Commits - feat(sidecar)!: introduce TraceData to unify text and binary data (#1247) - chore: add changelog for every published crate (#1396) ## libdd-data-pipeline **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-data-pipeline-v2.0.0` ### Commits - refactor(ddcommon)!: remove direct dependency on hyper client everywhere in common (#1604) - refactor(trace_exporter)!: remove Proxy TraceExporter input mode (#1583) - refactor(libdd-data-pipeline): health metrics (#1433) - feat(data-pipeline)!: include reason for chunks dropped telemetry (#1449) - feat(sidecar)!: introduce TraceData to unify text and binary data (#1247) - chore: release libddcommon-v1.1.0 (#1456) - chore: prepare libdd-telemetry-v2.0.0 (#1457) - Allow submitting Vec<Vec<Span>> asynchronously (#1302) - test(data-pipeline): handle EINTR in test_health_metrics_disabled (#1430) - chore: add changelog for every published crate (#1396) [APMSP-2590]: https://datadoghq.atlassian.net/browse/APMSP-2590?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>
What does this PR do?
Unifies the headers between common and profiling
Motivation
We had missed some when we implemented our own exporter, this makes sure the two implementations stay in sync.
Additional Notes
Anything else we should know when reviewing?
How to test the change?
Describe here in detail how the change can be validated.