feat: reduce telemetry sent by Leiyks · Pull Request #3316 · DataDog/dd-trace-php

Leiyks · 2025-06-25T13:52:43Z

Reduce telemetry volume sent to be a 30 min interval as precise application lifecycle tracking is not needed.
The goal is to improve performance as well as massively reduce the intake costs for PHP telemetry.

Related Jiras: LANGPLAT-395

pr-commenter · 2025-07-07T15:57:32Z

Benchmarks [ tracer ]

Benchmark execution time: 2025-07-28 17:33:07

Comparing candidate commit ca9c597 in PR branch leiyks-reduce-telemetry-size with baseline commit 3a60489 in branch master.

Found 15 performance improvements and 6 performance regressions! Performance is the same for 157 metrics, 0 unstable metrics.

scenario:EmptyFileBench/benchEmptyFileBaseline

🟩 execution_time [-747.904µs; -485.176µs] or [-19.203%; -12.457%]

scenario:EmptyFileBench/benchEmptyFileBaseline-opcache

🟩 execution_time [-718.580µs; -461.120µs] or [-17.917%; -11.498%]

scenario:EmptyFileBench/benchEmptyFileOverhead

🟩 execution_time [-766.160µs; -548.400µs] or [-18.070%; -12.934%]

scenario:EmptyFileBench/benchEmptyFileOverhead-opcache

🟩 execution_time [-775.887µs; -545.413µs] or [-17.844%; -12.544%]

scenario:LaravelBench/benchLaravelBaseline

🟩 execution_time [-581.994µs; -446.226µs] or [-5.103%; -3.913%]

scenario:LaravelBench/benchLaravelBaseline-opcache

🟩 execution_time [-663.218µs; -396.762µs] or [-5.712%; -3.417%]

scenario:LaravelBench/benchLaravelOverhead

🟩 execution_time [-1.698ms; -1.513ms] or [-12.634%; -11.256%]

scenario:LaravelBench/benchLaravelOverhead-opcache

🟩 execution_time [-1.626ms; -1.528ms] or [-12.042%; -11.317%]

scenario:PDOBench/benchPDOOverhead

🟥 execution_time [+117.923µs; +124.700µs] or [+31.838%; +33.667%]

scenario:PDOBench/benchPDOOverhead-opcache

🟥 execution_time [+117.932µs; +122.329µs] or [+32.346%; +33.552%]

scenario:PDOBench/benchPDOOverheadWithDBM

🟥 execution_time [+122.618µs; +128.087µs] or [+33.470%; +34.963%]

scenario:PDOBench/benchPDOOverheadWithDBM-opcache

🟥 execution_time [+117.951µs; +123.482µs] or [+32.271%; +33.785%]

scenario:PHPRedisBench/benchRedisOverhead

🟥 execution_time [+577.581µs; +614.551µs] or [+36.965%; +39.331%]

scenario:PHPRedisBench/benchRedisOverhead-opcache

🟥 execution_time [+594.398µs; +618.847µs] or [+37.385%; +38.923%]

scenario:SymfonyBench/benchSymfonyBaseline

🟩 execution_time [-538.550µs; -453.030µs] or [-7.320%; -6.158%]

scenario:SymfonyBench/benchSymfonyBaseline-opcache

🟩 execution_time [-545.231µs; -492.069µs] or [-7.262%; -6.554%]

scenario:SymfonyBench/benchSymfonyOverhead

🟩 execution_time [-1.115ms; -1.024ms] or [-12.916%; -11.857%]

scenario:SymfonyBench/benchSymfonyOverhead-opcache

🟩 execution_time [-1.051ms; -0.984ms] or [-12.015%; -11.246%]

scenario:TraceAnnotationsBench/benchTraceAnnotationOverhead

🟩 execution_time [-9.956µs; -4.379µs] or [-4.585%; -2.017%]

scenario:WordPressBench/benchWordPressOverhead

🟩 execution_time [-6.059ms; -5.860ms] or [-17.772%; -17.187%]

scenario:WordPressBench/benchWordPressOverhead-opcache

🟩 execution_time [-5.991ms; -5.768ms] or [-17.498%; -16.846%]

codecov-commenter · 2025-07-11T12:10:09Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.82%. Comparing base (3a60489) to head (232cf5e).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3316      +/-   ##
==========================================
- Coverage   61.93%   61.82%   -0.12%     
==========================================
  Files         140      140              
  Lines       12356    12356              
  Branches     1616     1616              
==========================================
- Hits         7653     7639      -14     
- Misses       3992     4005      +13     
- Partials      711      712       +1

see 2 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a60489...232cf5e. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Alexandre Rulleau <[email protected]>

Signed-off-by: Bob Weinand <[email protected]>

bwoebi

Nice - I fixed a windows issue, but now it compiles. Merging.

The sidecar's write_shm_file() serializes buffered_integrations as HashSet<Integration>, but the PHP-side reader in components-rs deserialized it as HashSet<String>. This mismatch has existed since the SHM cache was introduced in 91222ad ("feat: reduce telemetry sent", #3316), but was latent: config_sent was set true before any integrations appeared in SHM, and integration dedup (the only other consumer) was silently broken with the sidecar deduplicating server-side as a fallback. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* Batch endpoint collection Signed-off-by: Bob Weinand <[email protected]> * Avoid duplicate Symfony endpoint collections & don't collect every page on wordpress Batch ack sending too Signed-off-by: Bob Weinand <[email protected]> * Match SHM cache Integration type with sidecar writer The sidecar's write_shm_file() serializes buffered_integrations as HashSet<Integration>, but the PHP-side reader in components-rs deserialized it as HashSet<String>. This mismatch has existed since the SHM cache was introduced in 91222ad ("feat: reduce telemetry sent", #3316), but was latent: config_sent was set true before any integrations appeared in SHM, and integration dedup (the only other consumer) was silently broken with the sidecar deduplicating server-side as a fallback. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * Use a proper struct for Telemetry SHM data Signed-off-by: Bob Weinand <[email protected]> * Update Makefile Signed-off-by: Bob Weinand <[email protected]> --------- Signed-off-by: Bob Weinand <[email protected]> Co-authored-by: Gustavo André dos Santos Lopes <[email protected]> Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

Leiyks force-pushed the leiyks-reduce-telemetry-size branch 6 times, most recently from c6f22c6 to c8b809d Compare July 2, 2025 13:09

Leiyks force-pushed the leiyks-reduce-telemetry-size branch 7 times, most recently from 80e8052 to 85e17b7 Compare July 7, 2025 15:04

Leiyks force-pushed the leiyks-reduce-telemetry-size branch 4 times, most recently from f26f827 to 5b48ae8 Compare July 9, 2025 15:50

Leiyks marked this pull request as ready for review July 9, 2025 15:57

Leiyks requested review from a team as code owners July 9, 2025 15:57

bwoebi reviewed Jul 9, 2025

View reviewed changes

Comment thread components-rs/telemetry.rs Outdated

bwoebi reviewed Jul 9, 2025

View reviewed changes

Comment thread ext/sidecar.c Outdated

bwoebi reviewed Jul 9, 2025

View reviewed changes

Comment thread ext/telemetry.c Outdated

bwoebi reviewed Jul 11, 2025

View reviewed changes

Comment thread components-rs/telemetry.rs Outdated

bwoebi reviewed Jul 11, 2025

View reviewed changes

Comment thread components-rs/telemetry.rs Outdated

bwoebi reviewed Jul 11, 2025

View reviewed changes

Comment thread ext/sidecar.c Outdated

Leiyks force-pushed the leiyks-reduce-telemetry-size branch from dd69310 to 478f8d1 Compare July 11, 2025 14:55

Leiyks force-pushed the leiyks-reduce-telemetry-size branch 3 times, most recently from 7e74454 to 4edabe3 Compare July 23, 2025 15:03

Leiyks added 11 commits July 24, 2025 11:05

feat: update libdatadog version

b90e809

Signed-off-by: Alexandre Rulleau <[email protected]>

feat: update telemetry calls

690581a

Signed-off-by: Alexandre Rulleau <[email protected]>

feat(ext): move filtering code to Rust side

6aab05e

Signed-off-by: Alexandre Rulleau <[email protected]>

chore: fix tests

ce39a75

Signed-off-by: Alexandre Rulleau <[email protected]>

chore: temporary deactivate flaky test

75a8bfd

Signed-off-by: Alexandre Rulleau <[email protected]>

chore: fix segfault

0a15e98

Signed-off-by: Alexandre Rulleau <[email protected]>

chore: update libdatadog

b529f98

Signed-off-by: Alexandre Rulleau <[email protected]>

chore: clean code

234eb80

Signed-off-by: Alexandre Rulleau <[email protected]>

test: update integration tests

140caae

Signed-off-by: Alexandre Rulleau <[email protected]>

feat: remove needs for version

903aaa4

Signed-off-by: Alexandre Rulleau <[email protected]>

feat: add telemetry caching system for SHM read operation

84f4c78

Signed-off-by: Alexandre Rulleau <[email protected]>

Leiyks force-pushed the leiyks-reduce-telemetry-size branch 5 times, most recently from 6f991dc to abb70e5 Compare July 25, 2025 13:02

feat(sidecar: telemetry): handle config buffering

232cf5e

Signed-off-by: Alexandre Rulleau <[email protected]>

Leiyks force-pushed the leiyks-reduce-telemetry-size branch from abb70e5 to 232cf5e Compare July 28, 2025 15:02

Fix windows compilation

ca9c597

Signed-off-by: Bob Weinand <[email protected]>

bwoebi approved these changes Jul 28, 2025

View reviewed changes

bwoebi merged commit 91222ad into master Jul 28, 2025
1832 of 1879 checks passed

bwoebi deleted the leiyks-reduce-telemetry-size branch July 28, 2025 20:15

github-actions Bot added this to the 1.11.0 milestone Jul 28, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: reduce telemetry sent#3316

feat: reduce telemetry sent#3316
bwoebi merged 13 commits intomasterfrom
leiyks-reduce-telemetry-size

Leiyks commented Jun 25, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Jul 7, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jul 11, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bwoebi left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Leiyks commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks [ tracer ]

scenario:EmptyFileBench/benchEmptyFileBaseline

scenario:EmptyFileBench/benchEmptyFileBaseline-opcache

scenario:EmptyFileBench/benchEmptyFileOverhead

scenario:EmptyFileBench/benchEmptyFileOverhead-opcache

scenario:LaravelBench/benchLaravelBaseline

scenario:LaravelBench/benchLaravelBaseline-opcache

scenario:LaravelBench/benchLaravelOverhead

scenario:LaravelBench/benchLaravelOverhead-opcache

scenario:PDOBench/benchPDOOverhead

scenario:PDOBench/benchPDOOverhead-opcache

scenario:PDOBench/benchPDOOverheadWithDBM

scenario:PDOBench/benchPDOOverheadWithDBM-opcache

scenario:PHPRedisBench/benchRedisOverhead

scenario:PHPRedisBench/benchRedisOverhead-opcache

scenario:SymfonyBench/benchSymfonyBaseline

scenario:SymfonyBench/benchSymfonyBaseline-opcache

scenario:SymfonyBench/benchSymfonyOverhead

scenario:SymfonyBench/benchSymfonyOverhead-opcache

scenario:TraceAnnotationsBench/benchTraceAnnotationOverhead

scenario:WordPressBench/benchWordPressOverhead

scenario:WordPressBench/benchWordPressOverhead-opcache

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bwoebi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Leiyks commented Jun 25, 2025 •

edited

Loading

pr-commenter Bot commented Jul 7, 2025 •

edited

Loading

codecov-commenter commented Jul 11, 2025 •

edited

Loading