Skip to content

Add support for endpoint_counts#4980

Merged
szegedi merged 8 commits into
masterfrom
szegedi/endpoint-counts
Dec 11, 2024
Merged

Add support for endpoint_counts#4980
szegedi merged 8 commits into
masterfrom
szegedi/endpoint-counts

Conversation

@szegedi

@szegedi szegedi commented Dec 9, 2024

Copy link
Copy Markdown
Contributor

What does this PR do?

The profiling UI can show a CPU - Average Time Per Call For Top Endpoints if it is given an endpoint_counts JSON property in the uploaded event.json. We now add this property.

Motivation

We want to get the Node.js functionality to match https://docs.datadoghq.com/profiler/guide/isolate-outliers-in-monolithic-services as much as possible.

Additional Notes

This will also need a change in logs_backend before it starts showing up.

JIRA: PROF-11013

@szegedi
szegedi requested a review from a team as a code owner December 9, 2024 10:55
@szegedi
szegedi marked this pull request as draft December 9, 2024 10:56
@github-actions

github-actions Bot commented Dec 9, 2024

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 8.22 MB
Deduped: 94.73 MB
No deduping: 95.29 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.2.2 | 29.27 MB | 29.27 MB | | @datadog/native-appsec | 8.3.0 | 19.37 MB | 19.38 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.5.0 | 2.51 MB | 2.65 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.0.1 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | msgpack-lite | 0.1.26 | 201.16 kB | 281.59 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | int64-buffer | 0.1.10 | 49.18 kB | 49.18 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@pr-commenter

pr-commenter Bot commented Dec 9, 2024

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2024-12-09 19:55:29

Comparing candidate commit 06f2eed in PR branch szegedi/endpoint-counts with baseline commit af176d1 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 259 metrics, 7 unstable metrics.

@szegedi
szegedi force-pushed the szegedi/endpoint-counts branch 2 times, most recently from 1c16c01 to fff0e1f Compare December 9, 2024 13:16
File exporter will be writing it so we can more easily write integration tests.
@szegedi
szegedi force-pushed the szegedi/endpoint-counts branch from fff0e1f to 2acbae0 Compare December 9, 2024 13:25
@codecov

codecov Bot commented Dec 9, 2024

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 79.24528% with 11 lines in your changes missing coverage. Please review.

Project coverage is 79.41%. Comparing base (823cfd4) to head (2acbae0).
Report is 6 commits behind head on master.

Files with missing lines Patch % Lines
packages/dd-trace/src/profiling/profiler.js 60.71% 11 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #4980       +/-   ##
===========================================
+ Coverage   65.05%   79.41%   +14.36%     
===========================================
  Files         304      346       +42     
  Lines       13950    15336     +1386     
===========================================
+ Hits         9075    12179     +3104     
+ Misses       4875     3157     -1718     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@szegedi
szegedi marked this pull request as ready for review December 9, 2024 13:41
nsavoire
nsavoire previously approved these changes Dec 9, 2024

@nsavoire nsavoire left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a few questions / nitpicks !

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice refactoring !

Comment thread packages/dd-trace/src/profiling/exporters/event_serializer.js Outdated
Comment thread packages/dd-trace/src/profiling/profiler.js Outdated
Comment thread packages/dd-trace/src/profiling/profiler.js Outdated
Comment thread packages/dd-trace/src/profiling/profiler.js
].filter(v => v).join(' ')
if (!endpointName) return

let counter = this.endpointCounts.get(endpointName)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we guarenteed that a web span cannot have another web span with the same endpoint as child ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my reading of the code, it doesn't seem to be the case, but I can ask around on the guild

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some code to check if this is the outermost web span

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a quick experiment with a simple nextjs app and indeed we get to 2 web spans with same endpoint per request, probably one from http plugin and one from nextjs plugin.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for checking this!

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remembered seeing some weird things with nextjs when I added endpoint code 😄
I read the conversation about endpoint determination on slack, and I agree we need to come up with a cleaner way to gather the information in the tracer.

@szegedi
szegedi force-pushed the szegedi/endpoint-counts branch from 374a519 to 3420111 Compare December 9, 2024 19:41
@szegedi
szegedi requested a review from nsavoire December 9, 2024 20:00
@szegedi
szegedi merged commit 50bb0dd into master Dec 11, 2024
@szegedi
szegedi deleted the szegedi/endpoint-counts branch December 11, 2024 11:02
@rochdev rochdev mentioned this pull request Dec 17, 2024
rochdev pushed a commit that referenced this pull request Dec 17, 2024
Also:

* Extract event.json creation in profile exporters so it can be shared between all exporters.
File exporter will be writing it so we can more easily write integration tests.

* Extract web span handling in profiler
@rochdev rochdev mentioned this pull request Dec 17, 2024
rochdev pushed a commit that referenced this pull request Dec 17, 2024
Also:

* Extract event.json creation in profile exporters so it can be shared between all exporters.
File exporter will be writing it so we can more easily write integration tests.

* Extract web span handling in profiler
rochdev pushed a commit that referenced this pull request Dec 18, 2024
Also:

* Extract event.json creation in profile exporters so it can be shared between all exporters.
File exporter will be writing it so we can more easily write integration tests.

* Extract web span handling in profiler
rochdev pushed a commit that referenced this pull request Dec 18, 2024
Also:

* Extract event.json creation in profile exporters so it can be shared between all exporters.
File exporter will be writing it so we can more easily write integration tests.

* Extract web span handling in profiler
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants