Partially upgrade instrumentation telemetry from v1 to v2 by ida613 · Pull Request #3827 · DataDog/dd-trace-js

ida613 · 2023-11-27T21:31:57Z

What does this PR do?

The original PR to upgrade telemetry broke too many things due to config file changes. Since the config file changes were made in order to support identifying config value origins in telemetry event app-started, we are putting it as "known" for now. We want to upgrade to v2 asap, and then add in origin for config values.

Additional Notes

Security

Datadog employees:

If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
This PR doesn't touch any of that.

Unsure? Have a question? Request a review!

Co-authored-by: Thomas Hunter II <[email protected]>

…js into instel_partial_upgrade

iunanua · 2023-12-06T17:55:14Z

Please note that there are differences in the telemetry logs v1 and v2 schemas.

In the v1, the payload is an array of log objects and in the v2 the payload is an object with a logs property which is the array of log objects

I don't know what would happen if we keep sending the old schema to the v2 api...

mehulsonowal · 2023-12-06T18:53:27Z

      logCollection: isTrue(DD_TELEMETRY_LOG_COLLECTION_ENABLED),
-      metrics: isTrue(DD_TELEMETRY_METRICS_ENABLED)
+      metrics: isTrue(DD_TELEMETRY_METRICS_ENABLED),
+      dependencyCollection: DD_TELEMETRY_DEPENDENCY_COLLECTION_ENABLED


Just to clarify cause this question came up with Ruby; if DD_INSTRUMENTATION_TELEMETRY_ENABLED=false, none of the other DD_TELEMETRY flags should trigger any sort of telemetry collection even if they are true.

mehulsonowal · 2023-12-06T19:10:56Z

  if (agentError) {
    warn('DATADOG TRACER DIAGNOSTIC - Agent Error: ' + agentError.message)
+    errors.agentError = {
+      code: agentError.code ? agentError.code : '',


What are the agent error codes based on? Does it follow the descriptions on this sheet?

it probably doesnt. so I'll remove it for now since it's optional, will try to add it in later

mehulsonowal · 2023-12-07T19:48:38Z

+  const app = {
+    products: getProducts(config),
+    configuration: flatten(config),
    additional_payload: []


we've pretty much deprecated additional_payload and we don't do anything with the data, I think you should be able to just delete it from here if you're not using it either

I'll remove it

* adding required headers for in tel * adding required headers for in tel * updating version in unit tests * updating version in unit tests * app_started- extended_heartbeat- header- * updating heartbeat and adding extendedheartbeat * updating telemetry to v2 * worked on addressing telemetry uniform heartbeat * added some tests for instel v2 * cleaned up instel tests * adding dependencies variable check & tests * change to capture initial load for dependencies * Initial retry logic for telemetry * fixing heartbeat tests * adding in extended heartbeat * Cleaned up Index and send data * fixed syntax on telemetry tests * Added extended heartbeat logic * updated app-heartbeat test * modified config file for app-started * updating extended heartbeat to setInterval * adding tests for extendedheartbeat * updating extended heartbeat to setInterval * adding tests for extendedheartbeat * fix app-dependencies-loaded * modified config file to support telemetry app-started * modified config file for telemetry * modified config file to report tracer config values to telemetry * finished updating telemetry event app-started from v1 to v2 * fixed config file * fixed lint errors * attempt to remove yarn.lock changes * fixed telemetry settimeout bug * removed config file changes for now * reverted changes in config.spec.js * fixed bugs * disabled telemetry app-heartbeat test for now * added in origin field for telemetry app-started config * only send integrations through telemetry when there are integrations to be sent * only send integrations through telemetry when there are integrations to be sent * corrected telemetry event name * added back missing telemetry config value * added commit to skip telemetry heartbeat test for now * fixed telemetry send-data * Update packages/dd-trace/src/telemetry/dependencies.js Co-authored-by: Thomas Hunter II <[email protected]> * Update packages/dd-trace/src/telemetry/dependencies.js Co-authored-by: Thomas Hunter II <[email protected]> * Update packages/dd-trace/src/telemetry/index.js Co-authored-by: Thomas Hunter II <[email protected]> * Update packages/dd-trace/src/telemetry/index.js Co-authored-by: Thomas Hunter II <[email protected]> * Update packages/dd-trace/src/telemetry/index.js Co-authored-by: Thomas Hunter II <[email protected]> * Update packages/dd-trace/src/telemetry/send-data.js Co-authored-by: Thomas Hunter II <[email protected]> * Update packages/dd-trace/src/telemetry/index.js Co-authored-by: Thomas Hunter II <[email protected]> * added agentless telemetry data center routing * modified agentless telemetry routing * updated telemetry logs schema * polished instrumentation telemetry PR * removed app-started error field for now --------- Co-authored-by: Crystal Magloire <[email protected]> Co-authored-by: Sam Brenner <[email protected]> Co-authored-by: Sam Brenner <[email protected]> Co-authored-by: bojbrook <[email protected]> Co-authored-by: Crystal Magloire <[email protected]> Co-authored-by: Thomas Hunter II <[email protected]>

crysmags and others added 30 commits August 3, 2023 10:15

adding required headers for in tel

67070f8

adding required headers for in tel

1c1b689

updating version in unit tests

40120d3

updating version in unit tests

e3cbf77

app_started- extended_heartbeat- header-

86a8d1e

updating heartbeat and adding extendedheartbeat

8ecc43c

updating telemetry to v2

fd4f5ba

worked on addressing telemetry uniform heartbeat

9b1a184

added some tests for instel v2

60182db

Merge branch 'master' into inTelHeaders

dd61787

cleaned up instel tests

f14bc60

adding dependencies variable check & tests

a3e4032

Merge branch 'master' into inTelHeaders

1cec111

Merge branch 'master' into inTelHeaders

1c89df2

Merge branch 'master' into inTelHeaders

e09f43e

change to capture initial load for dependencies

a8b160e

Initial retry logic for telemetry

04ca3d9

fixing heartbeat tests

6ae6344

adding in extended heartbeat

9f42c77

Cleaned up Index and send data

6eaa7bc

Merge branch 'inTelHeaders' into telemetry-message-retry

4e2a6a6

fixed syntax on telemetry tests

3a2a730

Added extended heartbeat logic

0053707

Merge branch 'master' into telemetry-message-retry

9373a43

updated app-heartbeat test

17dc15b

modified config file for app-started

9b280bd

Merge branch 'telemetry-message-retry' into inTelHeaders

615ef10

Merge branch 'master' into inTelHeaders

03aa25b

updating extended heartbeat to setInterval

ff9fc0a

adding tests for extendedheartbeat

d3a36a2

ida613 and others added 4 commits December 4, 2023 15:25

Update packages/dd-trace/src/telemetry/send-data.js

a5b0b01

Co-authored-by: Thomas Hunter II <[email protected]>

Update packages/dd-trace/src/telemetry/index.js

a91cc03

Co-authored-by: Thomas Hunter II <[email protected]>

added agentless telemetry data center routing

fd1cb1d

Merge branch 'instel_partial_upgrade' of github.com:DataDog/dd-trace-…

2ad2b5b

…js into instel_partial_upgrade

github-advanced-security AI found potential problems Dec 6, 2023

View reviewed changes

Comment thread packages/dd-trace/src/telemetry/send-data.js Fixed

Comment thread packages/dd-trace/src/telemetry/send-data.js Fixed

ida613 added 3 commits December 6, 2023 13:21

modified agentless telemetry routing

4f0a660

updated telemetry logs schema

5aea9b9

Merge branch 'master' into instel_partial_upgrade

ec8a110

mehulsonowal reviewed Dec 8, 2023

View reviewed changes

ida613 added 3 commits December 11, 2023 14:16

polished instrumentation telemetry PR

85d7046

Merge branch 'master' into instel_partial_upgrade

f835715

removed app-started error field for now

119bb1e

tlhunter approved these changes Dec 11, 2023

View reviewed changes

ida613 merged commit 1adcae5 into master Dec 11, 2023

This was referenced Dec 13, 2023

V3.42.0 proposal #3869

Merged

V4.21.0 proposal #3871

Merged

tlhunter deleted the instel_partial_upgrade branch January 19, 2024 22:19

iunanua mentioned this pull request Feb 20, 2024

Re-enable telemetry logs #4079

Merged

8 tasks

erikvanbrakel mentioned this pull request Jul 5, 2024

[DEVX-3565] add neo4j plugin for v4.41 mazedesignhq/dd-trace-js#15

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Partially upgrade instrumentation telemetry from v1 to v2#3827

Partially upgrade instrumentation telemetry from v1 to v2#3827
ida613 merged 77 commits into
masterfrom
instel_partial_upgrade

ida613 commented Nov 27, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

iunanua commented Dec 6, 2023

Uh oh!

mehulsonowal Dec 6, 2023

Uh oh!

mehulsonowal Dec 6, 2023

Uh oh!

ida613 Dec 11, 2023

Uh oh!

mehulsonowal Dec 7, 2023

Uh oh!

ida613 Dec 8, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Uh oh!

Conversation

ida613 commented Nov 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Additional Notes

Security

Uh oh!

Uh oh!

Uh oh!

iunanua commented Dec 6, 2023

Uh oh!

mehulsonowal Dec 6, 2023

Choose a reason for hiding this comment

Uh oh!

mehulsonowal Dec 6, 2023

Choose a reason for hiding this comment

Uh oh!

ida613 Dec 11, 2023

Choose a reason for hiding this comment

Uh oh!

mehulsonowal Dec 7, 2023

Choose a reason for hiding this comment

Uh oh!

ida613 Dec 8, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

ida613 commented Nov 27, 2023 •

edited

Loading