Normalize the environment variable names used by crashtracking by kevingosse · Pull Request #5898 · DataDog/dd-trace-dotnet

kevingosse · 2024-08-14T12:44:43Z

Summary of changes

Normalize the environment variable names used by crashtracking.

DD_CRASHTRACKING_*: environment variables expected to be set by the customers. Currently:

DD_CRASHTRACKING_ENABLED: enables or disables crashtracking (default: enabled)

DD_INTERNAL_CRASHTRACKING_*: environment variables used by the tracer infrastructure and/or tests. Currently:

DD_INTERNAL_CRASHTRACKING_PASSTHROUGH: automatically set to decide whether the real createdump should be called or not
DD_INTERNAL_CRASHTRACKING_OUTPUT: save the crash report to a file instead of using telemetry

Reason for change

Now that other languages are implementing the feature, it's important to have consistent names.

Implementation details

DD_TRACE_CRASH_HANDLER is not needed anymore, removed it.
Removed the profiler tests that relied on DD_TRACE_CRASH_HANDLER (they're not needed now that we have crashtracking tests).

Test coverage

The existing tests were updated.

andrewlock · 2024-08-14T13:11:58Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

datadog-ddstaging · 2024-08-14T13:16:34Z

Datadog Report

Branch report: kevin/crashtracking_env
Commit report: aa6d53e
Test service: dd-trace-dotnet

✅ 0 Failed, 348156 Passed, 2084 Skipped, 16h 12m 58.65s Total Time

andrewlock · 2024-08-14T14:59:14Z

Benchmarks Report for tracer 🐌

Benchmarks for #5898 compared to master:

1 benchmarks are faster, with geometric mean 1.133
1 benchmarks are slower, with geometric mean 1.283
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.52μs	41.3ns	261ns	0.0189	0.00755	0	5.43 KB
master	`StartStopWithChild`	netcoreapp3.1	9.96μs	53.9ns	290ns	0.0196	0.00978	0	5.61 KB
master	`StartStopWithChild`	net472	16.1μs	44.4ns	172ns	1.01	0.291	0.0889	6.07 KB
#5898	`StartStopWithChild`	net6.0	7.69μs	43.2ns	299ns	0.0117	0.0039	0	5.43 KB
#5898	`StartStopWithChild`	netcoreapp3.1	9.92μs	54.9ns	320ns	0.0198	0.00991	0	5.61 KB
#5898	`StartStopWithChild`	net472	16.5μs	72.2ns	280ns	1.01	0.302	0.098	6.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	473μs	129ns	484ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	628μs	211ns	789ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	853μs	365ns	1.41μs	0.425	3.3 KB
#5898	`WriteAndFlushEnrichedTraces`	net6.0	472μs	213ns	823ns	0	2.7 KB
#5898	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	637μs	376ns	1.41μs	0	2.7 KB
#5898	`WriteAndFlushEnrichedTraces`	net472	852μs	423ns	1.64μs	0.419	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	193μs	1.09μs	7.24μs	0.183	18.45 KB
master	`SendRequest`	netcoreapp3.1	214μs	1.21μs	8.81μs	0.2	20.61 KB
master	`SendRequest`	net472	5.87E‑05ns	5.87E‑05ns	0.000212ns	0	0 b
#5898	`SendRequest`	net6.0	190μs	1.07μs	7.19μs	0.191	18.45 KB
#5898	`SendRequest`	netcoreapp3.1	214μs	1.2μs	8.15μs	0.206	20.61 KB
#5898	`SendRequest`	net472	0.000444ns	0.000164ns	0.000566ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	548μs	1.6μs	5.99μs	0.561	0	0	41.59 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	689μs	3.79μs	22.4μs	0.34	0	0	41.71 KB
master	`WriteAndFlushEnrichedTraces`	net472	868μs	3.25μs	11.7μs	8.3	2.62	0.437	53.3 KB
#5898	`WriteAndFlushEnrichedTraces`	net6.0	549μs	2.22μs	8.02μs	0.573	0	0	41.57 KB
#5898	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	670μs	3.34μs	14.6μs	0.331	0	0	41.63 KB
#5898	`WriteAndFlushEnrichedTraces`	net472	865μs	3.54μs	13.7μs	8.25	2.6	0.434	53.28 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.28μs	1.48ns	5.72ns	0.0142	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.72μs	1.54ns	5.98ns	0.0136	1.02 KB
master	`ExecuteNonQuery`	net472	1.98μs	1.53ns	5.94ns	0.156	987 B
#5898	`ExecuteNonQuery`	net6.0	1.32μs	0.895ns	3.47ns	0.0144	1.02 KB
#5898	`ExecuteNonQuery`	netcoreapp3.1	1.71μs	1.68ns	6.52ns	0.0136	1.02 KB
#5898	`ExecuteNonQuery`	net472	1.96μs	1.55ns	5.78ns	0.157	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.22μs	0.885ns	3.31ns	0.0134	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.54μs	1.19ns	4.46ns	0.0133	976 B
master	`CallElasticsearch`	net472	2.51μs	9.7ns	36.3ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.33μs	0.986ns	3.69ns	0.0133	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.68μs	0.944ns	3.53ns	0.0137	1.02 KB
master	`CallElasticsearchAsync`	net472	2.6μs	4.59ns	17.8ns	0.166	1.05 KB
#5898	`CallElasticsearch`	net6.0	1.1μs	0.389ns	1.45ns	0.0138	976 B
#5898	`CallElasticsearch`	netcoreapp3.1	1.54μs	0.562ns	2.1ns	0.0132	976 B
#5898	`CallElasticsearch`	net472	2.44μs	2.73ns	10.6ns	0.157	995 B
#5898	`CallElasticsearchAsync`	net6.0	1.2μs	0.714ns	2.77ns	0.0132	952 B
#5898	`CallElasticsearchAsync`	netcoreapp3.1	1.53μs	0.648ns	2.43ns	0.0139	1.02 KB
#5898	`CallElasticsearchAsync`	net472	2.64μs	2.4ns	9.28ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.34μs	0.737ns	2.85ns	0.0134	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.72μs	0.568ns	2.12ns	0.0129	952 B
master	`ExecuteAsync`	net472	1.75μs	1.31ns	4.89ns	0.145	915 B
#5898	`ExecuteAsync`	net6.0	1.25μs	1.12ns	4.33ns	0.0132	952 B
#5898	`ExecuteAsync`	netcoreapp3.1	1.57μs	0.669ns	2.5ns	0.0126	952 B
#5898	`ExecuteAsync`	net472	1.71μs	1.07ns	4.13ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.34μs	2.45ns	9.5ns	0.0304	2.22 KB
master	`SendAsync`	netcoreapp3.1	5.1μs	2.27ns	8.48ns	0.038	2.76 KB
master	`SendAsync`	net472	7.86μs	1.64ns	6.14ns	0.5	3.15 KB
#5898	`SendAsync`	net6.0	4.22μs	1.36ns	5.08ns	0.0315	2.22 KB
#5898	`SendAsync`	netcoreapp3.1	5.14μs	2.42ns	8.73ns	0.0359	2.76 KB
#5898	`SendAsync`	net472	7.77μs	1.6ns	6.18ns	0.498	3.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.47μs	0.572ns	2.14ns	0.0228	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.42μs	1.18ns	4.56ns	0.0216	1.64 KB
master	`EnrichedLog`	net472	2.69μs	1.14ns	4.4ns	0.249	1.57 KB
#5898	`EnrichedLog`	net6.0	1.48μs	0.631ns	2.36ns	0.023	1.64 KB
#5898	`EnrichedLog`	netcoreapp3.1	2.26μs	2.29ns	8.88ns	0.0215	1.64 KB
#5898	`EnrichedLog`	net472	2.76μs	1.8ns	6.97ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	117μs	231ns	894ns	0.0579	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	121μs	318ns	1.19μs	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	252ns	976ns	0.676	0.225	4.46 KB
#5898	`EnrichedLog`	net6.0	116μs	293ns	1.13μs	0.0587	0	4.28 KB
#5898	`EnrichedLog`	netcoreapp3.1	117μs	168ns	650ns	0.058	0	4.28 KB
#5898	`EnrichedLog`	net472	146μs	161ns	622ns	0.659	0.22	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.15μs	0.726ns	2.81ns	0.0307	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.21μs	1.27ns	4.92ns	0.0296	2.2 KB
master	`EnrichedLog`	net472	4.92μs	1.4ns	5.44ns	0.32	2.02 KB
#5898	`EnrichedLog`	net6.0	3.12μs	0.894ns	3.1ns	0.0312	2.2 KB
#5898	`EnrichedLog`	netcoreapp3.1	4.24μs	2.2ns	7.93ns	0.0296	2.2 KB
#5898	`EnrichedLog`	net472	4.99μs	1.26ns	4.73ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.35μs	0.913ns	3.54ns	0.0156	0	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.85μs	1.21ns	4.67ns	0.0156	0	1.14 KB
master	`SendReceive`	net472	2.18μs	1.75ns	6.79ns	0.183	0	1.16 KB
#5898	`SendReceive`	net6.0	1.34μs	0.623ns	2.41ns	0.0161	0	1.14 KB
#5898	`SendReceive`	netcoreapp3.1	1.75μs	0.748ns	2.8ns	0.0147	0	1.14 KB
#5898	`SendReceive`	net472	2.21μs	2.86ns	11.1ns	0.184	0.0011	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.86μs	0.697ns	2.61ns	0.0215	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.77μs	1.42ns	5.3ns	0.0207	1.65 KB
master	`EnrichedLog`	net472	4.5μs	3.2ns	12.4ns	0.322	2.04 KB
#5898	`EnrichedLog`	net6.0	2.73μs	0.857ns	3.32ns	0.0219	1.6 KB
#5898	`EnrichedLog`	netcoreapp3.1	3.91μs	1.71ns	6.64ns	0.0215	1.65 KB
#5898	`EnrichedLog`	net472	4.37μs	2.85ns	11ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5898

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472	1.133	634.79	560.43

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	399ns	0.0943ns	0.34ns	0.00803	576 B
master	`StartFinishSpan`	netcoreapp3.1	557ns	1.16ns	4.49ns	0.00796	576 B
master	`StartFinishSpan`	net472	635ns	0.427ns	1.65ns	0.0915	578 B
master	`StartFinishScope`	net6.0	493ns	1.19ns	4.62ns	0.00964	696 B
master	`StartFinishScope`	netcoreapp3.1	748ns	0.514ns	1.99ns	0.0094	696 B
master	`StartFinishScope`	net472	888ns	2.64ns	10.2ns	0.104	658 B
#5898	`StartFinishSpan`	net6.0	399ns	0.372ns	1.44ns	0.00819	576 B
#5898	`StartFinishSpan`	netcoreapp3.1	622ns	0.485ns	1.82ns	0.00777	576 B
#5898	`StartFinishSpan`	net472	560ns	0.362ns	1.25ns	0.0918	578 B
#5898	`StartFinishScope`	net6.0	482ns	0.515ns	1.99ns	0.00978	696 B
#5898	`StartFinishScope`	netcoreapp3.1	766ns	0.409ns	1.47ns	0.00933	696 B
#5898	`StartFinishScope`	net472	884ns	0.945ns	3.66ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5898

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.283	594.61	763.02

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	595ns	0.204ns	0.765ns	0.00967	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	966ns	0.499ns	1.93ns	0.00923	696 B
master	`RunOnMethodBegin`	net472	1.11μs	0.454ns	1.57ns	0.104	658 B
#5898	`RunOnMethodBegin`	net6.0	763ns	0.486ns	1.88ns	0.00972	696 B
#5898	`RunOnMethodBegin`	netcoreapp3.1	911ns	1.29ns	5.01ns	0.0092	696 B
#5898	`RunOnMethodBegin`	net472	1.06μs	0.501ns	1.88ns	0.104	658 B

chrisnas

LGTM with a couple of comments

chrisnas · 2024-08-16T10:11:57Z

        {
            // If DOTNET_DbgEnableMiniDump is set, the crash handler should call createdump when done
-            char* passthrough = getenv("DD_TRACE_CRASH_HANDLER_PASSTHROUGH");
+            char* passthrough = getenv("DD_INTERNAL_CRASHTRACKING_PASSTHROUGH");


Use constants for env var names to avoid repeating them and maybe introduce typos

Done in 904b7eb

chrisnas · 2024-08-16T10:44:58Z

-        }
-
-        [TestAppFact("Samples.ExceptionGenerator")]
-        public void DontRedirectCrashHandlerIfPathNotSet(string appName, string framework, string appAssembly)


Maybe keep this test and rename it into GenerateDumpIfDbgRequested to validate that existing CLR behavior is still working with CrashTracking

Done in 904b7eb

Normalize the environment variable names used by crashtracking. `DD_CRASHTRACKING_*`: environment variables expected to be set by the customers. Currently: - `DD_CRASHTRACKING_ENABLED`: enables or disables crashtracking (default: enabled) `DD_INTERNAL_CRASHTRACKING_*`: environment variables used by the tracer infrastructure and/or tests. Currently: - `DD_INTERNAL_CRASHTRACKING_PASSTHROUGH`: automatically set to decide whether the real createdump should be called or not - `DD_INTERNAL_CRASHTRACKING_OUTPUT`: save the crash report to a file instead of using telemetry Now that other languages are implementing the feature, it's important to have consistent names. `DD_TRACE_CRASH_HANDLER` is not needed anymore, removed it. Removed the profiler tests that relied on `DD_TRACE_CRASH_HANDLER` (they're not needed now that we have crashtracking tests). The existing tests were updated.

…-> v2) (#5936) ## Summary of changes Normalize the environment variable names used by crashtracking. `DD_CRASHTRACKING_*`: environment variables expected to be set by the customers. Currently: - `DD_CRASHTRACKING_ENABLED`: enables or disables crashtracking (default: enabled) `DD_INTERNAL_CRASHTRACKING_*`: environment variables used by the tracer infrastructure and/or tests. Currently: - `DD_INTERNAL_CRASHTRACKING_PASSTHROUGH`: automatically set to decide whether the real createdump should be called or not - `DD_INTERNAL_CRASHTRACKING_OUTPUT`: save the crash report to a file instead of using telemetry ## Reason for change Now that other languages are implementing the feature, it's important to have consistent names. ## Implementation details `DD_TRACE_CRASH_HANDLER` is not needed anymore, removed it. Removed the profiler tests that relied on `DD_TRACE_CRASH_HANDLER` (they're not needed now that we have crashtracking tests). ## Test coverage The existing tests were updated. ## Other Details Backport of - #5898 --------- Co-authored-by: Kevin Gosse <[email protected]>

Normalize the crashtracking environment variables

b3f8d8d

kevingosse requested review from a team as code owners August 14, 2024 12:44

andrewlock approved these changes Aug 14, 2024

View reviewed changes

TonyCTHsu approved these changes Aug 14, 2024

View reviewed changes

Remove DD_TRACE_CRASH_HANDLER tests

868c5db

Update functions_to_wrap.c

f74319d

chrisnas approved these changes Aug 16, 2024

View reviewed changes

Kevin Gosse added 2 commits August 16, 2024 15:40

Address PR feedback

904b7eb

Fix

aa6d53e

kevingosse merged commit ef38abe into master Aug 16, 2024

kevingosse deleted the kevin/crashtracking_env branch August 16, 2024 17:42

github-actions Bot added this to the vNext-v3 milestone Aug 16, 2024

andrewlock added the area:telemetry label Aug 23, 2024

andrewlock mentioned this pull request Aug 23, 2024

Normalize the environment variable names used by crashtracking (#5898 -> v2) #5936

Merged

andrewlock added the area:crashtracking label Oct 23, 2024

Uh oh!

Conversation

kevingosse commented Aug 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Test coverage

Uh oh!

andrewlock commented Aug 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

Uh oh!

datadog-ddstaging Bot commented Aug 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Datadog Report

Uh oh!

andrewlock commented Aug 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5898

Raw results

Slower ⚠️ in #5898

Raw results

Uh oh!

chrisnas left a comment

Choose a reason for hiding this comment

Uh oh!

chrisnas Aug 16, 2024

Choose a reason for hiding this comment

Uh oh!

kevingosse Aug 16, 2024

Choose a reason for hiding this comment

Uh oh!

chrisnas Aug 16, 2024

Choose a reason for hiding this comment

Uh oh!

kevingosse Aug 16, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kevingosse commented Aug 14, 2024 •

edited

Loading

andrewlock commented Aug 14, 2024 •

edited

Loading

datadog-ddstaging Bot commented Aug 14, 2024 •

edited

Loading

andrewlock commented Aug 14, 2024 •

edited

Loading