feat: otel process ctxt protobuf encoding#1651
feat: otel process ctxt protobuf encoding#1651gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits intomainfrom
Conversation
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
BenchmarksComparisonBenchmark execution time: 2026-03-05 15:43:30 Comparing candidate commit f8f7683 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 2 unstable metrics. CandidateCandidate benchmark detailsGroup 1
Group 2
Group 3
Group 4
Group 5
Group 6
Group 7
Group 8
Group 9
Group 10
Group 11
Group 12
Group 13
Group 14
Group 15
Group 16
Group 17
Group 18
Group 19
BaselineOmitted due to size. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1651 +/- ##
==========================================
- Coverage 71.24% 71.23% -0.01%
==========================================
Files 427 427
Lines 62805 62808 +3
==========================================
- Hits 44748 44744 -4
- Misses 18057 18064 +7
🚀 New features to boost your workflow:
|
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
dffff99 to
16cc9c4
Compare
a843b3c to
bd4ba00
Compare
ivoanjo
left a comment
There was a problem hiding this comment.
👍 LGTM
I was not sure where to put the protobuf definition, as there are a bunch of protobuf-dedicated crates in libdatadog. I feel like the OTel process context is about tracing metadata, but I'm happy to move it elsewhere if it makes more sense.
TBH I'd put this inside libdd-library-config, since it's an implementation detail for that crate, but no strong opinions and it's easy to move later if we want so as-is is fine.
de3ea26 to
2f163e6
Compare
bd4ba00 to
b696b76
Compare
2f163e6 to
9e85c64
Compare
12d43af to
25419ac
Compare
25419ac to
071be97
Compare
I think we will also need the |
Adds the process context protobuf definition, as well as a couple definitions taken from opentelemetry that the context depends on.
071be97 to
c3785c8
Compare
The previous interface for process context publication was rather low-level: the payload was taken as an arbitrary bag of bytes. However, the shape of the payload is known and constrained: it has a protobuf definition. This commit makes the interface higher-level by taking a structured process context instead, handling the protobuf encoding natively.
c3785c8 to
f8f7683
Compare
# What does this PR do? [feat: process context publication](#1585) [ci: pass macos label to downstream project](#1647) [chore: implement otel process ctx update](#1640) [chore(crashtracking): emit a best effort stacktrace for Mac](#1645) [ci: remove depth so it can cause problems when getting the diffs](#1657) [build(macOS): set the LC_ID_DYLIB for mac binaries to set correct name for linking](#1646) [chore(ci): fix crashtracker receiver binary rpath setting](#1652) [chore(deps): bump blazesym to 0.2.3 and blazesym-c to 0.1.7](#1654) [feat: otel process ctxt protobuf encoding](#1651) [chore(crashtracker): fix benchmark job](#1664) # Motivation What inspired you to submit this pull request? # Additional Notes Anything else we should know when reviewing? # How to test the change? Describe here in detail how the change can be validated. Co-authored-by: gyuheon.oh <[email protected]>
# Release proposal for libdd-library-config and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-trace-protobuf **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-protobuf-v2.0.0` ### Commits - feat(trace-protobuf)!: Add two fields to ClientGroupedStats [SVLS-8627] (#1630) - feat: otel process ctxt protobuf encoding (#1651) ## libdd-library-config **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-library-config-v1.1.0` ### Commits - chore: update otel process ctx protocol (#1713) - feat: publish tracer metadata as OTel process ctx (#1658) - feat: otel process ctxt protobuf encoding (#1651) - fix: handle fork in otel process ctx (#1650) - chore: implement otel process ctx update (#1640) - feat: process context publication (#1585) - ci: update nightly in CI to 2026-02-08 (#1539) - fix(stable_config): [APMAPI-1690] add >100mb check for stable config files (#1432) - chore: add changelog for every published crate (#1396) [SVLS-8627]: https://datadoghq.atlassian.net/browse/SVLS-8627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMAPI-1690]: https://datadoghq.atlassian.net/browse/APMAPI-1690?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>
# Release proposal for libdd-library-config and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-trace-protobuf **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-protobuf-v2.0.0` ### Commits - feat(trace-protobuf)!: Add two fields to ClientGroupedStats [SVLS-8627] (#1630) - feat: otel process ctxt protobuf encoding (#1651) ## libdd-library-config **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-library-config-v1.1.0` ### Commits - chore: update otel process ctx protocol (#1713) - feat: publish tracer metadata as OTel process ctx (#1658) - feat: otel process ctxt protobuf encoding (#1651) - fix: handle fork in otel process ctx (#1650) - chore: implement otel process ctx update (#1640) - feat: process context publication (#1585) - ci: update nightly in CI to 2026-02-08 (#1539) - fix(stable_config): [APMAPI-1690] add >100mb check for stable config files (#1432) - chore: add changelog for every published crate (#1396) [SVLS-8627]: https://datadoghq.atlassian.net/browse/SVLS-8627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMAPI-1690]: https://datadoghq.atlassian.net/browse/APMAPI-1690?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>
# Release proposal for libdd-library-config and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-trace-protobuf **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-protobuf-v2.0.0` ### Commits - feat(trace-protobuf)!: Add two fields to ClientGroupedStats [SVLS-8627] (#1630) - feat: otel process ctxt protobuf encoding (#1651) ## libdd-library-config **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-library-config-v1.1.0` ### Commits - chore: update otel process ctx protocol (#1713) - feat: publish tracer metadata as OTel process ctx (#1658) - feat: otel process ctxt protobuf encoding (#1651) - fix: handle fork in otel process ctx (#1650) - chore: implement otel process ctx update (#1640) - feat: process context publication (#1585) - ci: update nightly in CI to 2026-02-08 (#1539) - fix(stable_config): [APMAPI-1690] add >100mb check for stable config files (#1432) - chore: add changelog for every published crate (#1396) [SVLS-8627]: https://datadoghq.atlassian.net/browse/SVLS-8627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMAPI-1690]: https://datadoghq.atlassian.net/browse/APMAPI-1690?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- # What does this PR do? A brief description of the change being made with this pull request. # Motivation What inspired you to submit this pull request? # Additional Notes Anything else we should know when reviewing? # How to test the change? Describe here in detail how the change can be validated. [SVLS-8627]: https://datadoghq.atlassian.net/browse/SVLS-8627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMAPI-1690]: https://datadoghq.atlassian.net/browse/APMAPI-1690?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SVLS-8627]: https://datadoghq.atlassian.net/browse/SVLS-8627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>
# Release proposal for libdd-library-config and its dependencies This PR contains version bumps based on public API changes and commits since last release. ## libdd-trace-protobuf **Next version:** `2.0.0` **Semver bump:** `major` **Tag:** `libdd-trace-protobuf-v2.0.0` ### Commits - feat(trace-protobuf)!: Add two fields to ClientGroupedStats [SVLS-8627] (#1630) - feat: otel process ctxt protobuf encoding (#1651) ## libdd-library-config **Next version:** `1.1.0` **Semver bump:** `minor` **Tag:** `libdd-library-config-v1.1.0` ### Commits - chore: update otel process ctx protocol (#1713) - feat: publish tracer metadata as OTel process ctx (#1658) - feat: otel process ctxt protobuf encoding (#1651) - fix: handle fork in otel process ctx (#1650) - chore: implement otel process ctx update (#1640) - feat: process context publication (#1585) - ci: update nightly in CI to 2026-02-08 (#1539) - fix(stable_config): [APMAPI-1690] add >100mb check for stable config files (#1432) - chore: add changelog for every published crate (#1396) [SVLS-8627]: https://datadoghq.atlassian.net/browse/SVLS-8627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMAPI-1690]: https://datadoghq.atlassian.net/browse/APMAPI-1690?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>
Depends #1650
What does this PR do?
Follow-up of #1585 #1640 #1650
Adds the protobuf definition for the OTel process context and associated messages, and make the interface of the publisher higher-level by taking the new structured
ProcessContextvalue instead of a raw bytes payload.Motivation
Since libdatadog is already taking care of some protobuf encoding, and it's supposedly faster and simpler to do here rather than on the side of each language runtime, it makes sense to offer an interface with a struct that is encoded by libdatadog. The process context will be a
ProcessContext-based opaque pointer on the FFI side, with proper setters/getters.Additional Notes
I was not sure where to put the protobuf definition, as there are a bunch of protobuf-dedicated crates in libdatadog. I feel like the OTel process context is about tracing metadata, but I'm happy to move it elsewhere if it makes more sense.
How to test the change?
Once the FFI lands, we'll be able to publish the context from a language runtime and check the whole process end-to-end. For now, I feel like an additional test would mostly test
prost, which isn't very valuable.