[CI App] - Agentless Writer by tonyredondo · Pull Request #2402 · DataDog/dd-trace-dotnet

tonyredondo · 2022-02-04T10:06:01Z

The PR is primarily focused on introducing the V1 of Agentless test reporting on the .NET tracer as opt-in only when the CI Visibility mode is Enabled.

For the reviewing process, there's an internal document about all changes made by the PR.

@DataDog/apm-dotnet

lucaspimentel

Left a few comments but nothing that should block this PR.

andrewlock

Nice 👍

andrewlock · 2022-03-01T08:49:58Z

+                                return 1;
+                            }
+                        }
+                        else if (!Utils.CheckAgentConnectionAsync(options.AgentUrl).GetAwaiter().GetResult())


Do we want to/can we test the connection when using agentless mode? 🤔 May be a feature request for the backend, but would be useful to be able to check that a) connection is successful (presumably requires correct DD_SITE, haven't got that far yet) b) the API key is correct.

That can be helpful, I have to check with the backend team. Maybe for another PR.

andrewlock · 2022-03-01T09:38:52Z

+
+        // https://github.com/DataDog/datadog-agent/blob/eac2327c5574da7f225f9ef0f89eaeb05ed10382/pkg/trace/traceutil/truncate.go#L36-L51
+        [MethodImpl(MethodImplOptions.AggressiveInlining)]
+        public static bool TruncateUTF8(ref string value, int limit)


How strict is the tag length limit? What happens is we go over the limit, does it get truncated in the intake anyway or does it get rejected?

Only partially related, but if we use this logic to calculate metrics in the tracer (as we need to), then I think we need to make sure we use exactly the same truncation, otherwise we could end up with strange discrepancies? Not entirely sure or thought through yet though 😅

andrewlock · 2022-03-01T09:51:43Z

+            yield return new object[] { "#test_starting_hash", "test_starting_hash" };
+            yield return new object[] { "TestCAPSandSuch", "testcapsandsuch" };
+            yield return new object[] { "Test Conversion Of Weird !@#$%^&**() Characters", "test_conversion_of_weird_characters" };


You've triggered a new build since then, so you can't see the code coverage until the new one completely, but I think it would be good to check the code coverage for the TraceUtil class and make sure you've covered all the branches, given there's a lot of complexity in there! 🙂

I'll check, the tests are the same as the ones in the agent.

Co-authored-by: Andrew Lock <[email protected]>

…et into tony/agentless-poc

Co-authored-by: Andrew Lock <[email protected]>

…et into tony/agentless-poc

andrewlock · 2022-03-01T21:41:12Z

Benchmarks Report 🐌

Benchmarks for #2402 compared to master:

All benchmarks have the same speed
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`WriteAndFlushEnrichedTraces`	net472	913μs	5.03μs	29.8μs	3.16 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	689μs	3.81μs	22.8μs	2.57 KB
#2402	`WriteAndFlushEnrichedTraces`	net472	852μs	2.79μs	10.1μs	3.16 KB
#2402	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	661μs	3.18μs	12.3μs	2.57 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net472	0ns	0ns	0ns	0	0 b
master	`SendRequest`	netcoreapp3.1	295μs	1.53μs	7.02μs	0.291	19.71 KB
#2402	`SendRequest`	net472	0ns	0ns	0ns	0	0 b
#2402	`SendRequest`	netcoreapp3.1	294μs	902ns	3.5μs	0.146	19.71 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net472	1.7μs	6.33ns	24.5ns	0.0926	0.000835	586 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.48μs	6.19ns	24ns	0.0088	0	624 B
#2402	`ExecuteNonQuery`	net472	1.72μs	7.4ns	28.7ns	0.0927	0.000828	586 B
#2402	`ExecuteNonQuery`	netcoreapp3.1	1.52μs	6.55ns	25.3ns	0.00833	0	624 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`CallElasticsearch`	net472	2.8μs	15.3ns	90.6ns	0.125	0	0	794 B
master	`CallElasticsearch`	netcoreapp3.1	1.69μs	7.72ns	32.8ns	0.0111	0	0	784 B
master	`CallElasticsearchAsync`	net472	2.79μs	13.6ns	56.2ns	0.147	0.00137	0	931 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.87μs	10.8ns	83.7ns	0.0124	0	0	904 B
#2402	`CallElasticsearch`	net472	2.63μs	11ns	42.4ns	0.126	0	0	794 B
#2402	`CallElasticsearch`	netcoreapp3.1	1.75μs	9.49ns	56.2ns	0.0106	0	0	784 B
#2402	`CallElasticsearchAsync`	net472	2.8μs	13.2ns	55.9ns	0.147	0.00274	0.00137	931 B
#2402	`CallElasticsearchAsync`	netcoreapp3.1	2.01μs	3.84ns	14.4ns	0.0121	0	0	904 B

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteAsync`	net472	3.05μs	13.2ns	60.4ns	0.167	0.00157	1.05 KB
master	`ExecuteAsync`	netcoreapp3.1	2.03μs	8.87ns	40.7ns	0.0148	0	1.02 KB
#2402	`ExecuteAsync`	net472	3.1μs	16.7ns	86.6ns	0.166	0.00148	1.05 KB
#2402	`ExecuteAsync`	netcoreapp3.1	2.16μs	11.9ns	68.6ns	0.0137	0	1.02 KB

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net472	7.08μs	28.7ns	111ns	0.363	2.3 KB
master	`SendAsync`	netcoreapp3.1	4.89μs	17.5ns	67.9ns	0.0298	2.2 KB
#2402	`SendAsync`	net472	7.16μs	23.4ns	90.8ns	0.364	2.3 KB
#2402	`SendAsync`	netcoreapp3.1	4.81μs	14.4ns	53.7ns	0.0304	2.2 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net472	3.22μs	15ns	60.2ns	0.226	1.44 KB
master	`EnrichedLog`	netcoreapp3.1	3.1μs	10.2ns	39.5ns	0.0203	1.52 KB
#2402	`EnrichedLog`	net472	3.36μs	17.9ns	99.7ns	0.225	1.44 KB
#2402	`EnrichedLog`	netcoreapp3.1	3.13μs	11.7ns	43.8ns	0.0204	1.52 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net472	290μs	1.06μs	4.1μs	0.437	0.146	4.31 KB
master	`EnrichedLog`	netcoreapp3.1	231μs	793ns	2.97μs	0	0	4.19 KB
#2402	`EnrichedLog`	net472	292μs	1.42μs	6.04μs	0.434	0.145	4.31 KB
#2402	`EnrichedLog`	netcoreapp3.1	231μs	912ns	3.53μs	0	0	4.19 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net472	7.48μs	38.9ns	198ns	0.502	3.21 KB
master	`EnrichedLog`	netcoreapp3.1	6.43μs	12.4ns	48ns	0.0491	3.58 KB
#2402	`EnrichedLog`	net472	7.47μs	35ns	140ns	0.504	3.21 KB
#2402	`EnrichedLog`	netcoreapp3.1	6.47μs	28.1ns	105ns	0.0483	3.58 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net472	2.47μs	11.6ns	47.6ns	0.159	0.00123	1 KB
master	`SendReceive`	netcoreapp3.1	2.04μs	9.42ns	35.2ns	0.0137	0	1 KB
#2402	`SendReceive`	net472	2.54μs	14.1ns	90ns	0.157	0	1 KB
#2402	`SendReceive`	netcoreapp3.1	2.08μs	8.19ns	30.6ns	0.0138	0	1 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net472	6.42μs	35.9ns	238ns	0.288	1.85 KB
master	`EnrichedLog`	netcoreapp3.1	5.39μs	13.6ns	52.6ns	0.019	1.47 KB
#2402	`EnrichedLog`	net472	6.75μs	37.4ns	221ns	0.289	1.85 KB
#2402	`EnrichedLog`	netcoreapp3.1	5.45μs	31ns	221ns	0.0196	1.47 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net472	916ns	4.98ns	28.2ns	0.0703	449 B
master	`StartFinishSpan`	netcoreapp3.1	902ns	3.39ns	13.1ns	0.00633	448 B
master	`StartFinishScope`	net472	1.09μs	4.39ns	17ns	0.0832	530 B
master	`StartFinishScope`	netcoreapp3.1	1.07μs	5.08ns	21ns	0.00756	568 B
#2402	`StartFinishSpan`	net472	900ns	3.14ns	12.2ns	0.0703	449 B
#2402	`StartFinishSpan`	netcoreapp3.1	893ns	2.55ns	9.86ns	0.00613	448 B
#2402	`StartFinishScope`	net472	1.11μs	4.9ns	18.3ns	0.0832	530 B
#2402	`StartFinishScope`	netcoreapp3.1	1.06μs	4.96ns	19.2ns	0.0077	568 B

andrewlock · 2022-03-01T21:46:23Z

Code Coverage Report 📊

⚠️ Merging #2402 into master will will decrease line coverage by 3%
⚠️ Merging #2402 into master will will decrease branch coverage by 1%
⛔ Merging #2402 into master will will increase complexity by 443

	master	#2402	Change
Lines	`11650` / `15997`	`11785` / `16764`
Lines %	`73%`	`70%`	`-3%` ⚠️
Branches	`6261` / `9364`	`6385` / `9750`
Branches %	`67%`	`65%`	`-1%` ⚠️
Complexity	`10573`	`11016`	`443` ⛔

View the full report for further details:

Datadog.Trace Breakdown ⚠️

	master	#2402	Change
Lines %	`73%`	`70%`	`-3%` ⚠️
Branches %	`67%`	`65%`	`-1%` ⚠️
Complexity	`10573`	`11016`	`443` ⛔

The following classes have significant coverage changes.

File	Line coverage change	Branch coverage change	Complexity change
Datadog.Trace.Agent.StreamFactories.TcpStreamFactory	`-90%` ⛔	`0%` ✔️	`0` ✔️
Datadog.Trace.Ci.CITracerManagerFactory	`-76%` ⛔	`-90%` ⛔	`11` ⛔
Datadog.Trace.Ci.CITracerManager	`-70%` ⛔	`-100%` ⛔	`11` ⛔
Datadog.Trace.Ci.Agent.CIAgentWriter	`-50%` ⛔	`-93%` ⛔	`-6` ✔️
Datadog.Trace.Agent.Transports.ApiWebRequestFactory	`-36%` ⛔	`-33%` ⛔	`5` ⛔
Datadog.Trace.Agent.Transports.HttpClientRequestFactory	`-36%` ⛔	`-33%` ⛔	`2` ⛔
Datadog.Trace.Telemetry.JsonHttpClientTelemetryTransport	`-26%` ⛔	`-38%` ⛔	`0` ✔️
Datadog.Trace.Agent.TracesTransportStrategy	`-15%` ⛔	`0%` ✔️	`0` ✔️
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory	`-13%` ⛔	`0%` ✔️	`1` ⛔
Datadog.Trace.ClrProfiler.AutoInstrumentation.MongoDb.MongoDbIntegration	`-7%` ⛔	`0%` ✔️	`0` ✔️
...And 3 more

The following classes were added in #2402:

File	Line coverage	Branch coverage	Complexity
Datadog.Trace.Ci.Agent.CIAgentlessWriter	`0%`	`0%`	`40`
Datadog.Trace.Ci.Agent.CIWriterFileSender	`0%`	`100%`	`3`
Datadog.Trace.Ci.Agent.CIWriterHttpSender	`0%`	`0%`	`24`
Datadog.Trace.Ci.Agent.MessagePack.CIEventMessagePackFormatter	`0%`	`0%`	`20`
Datadog.Trace.Ci.Agent.MessagePack.CIFormatterResolver	`0%`	`0%`	`12`
...And 17 more

View the full reports for further details:

tonyredondo added 30 commits December 13, 2021 22:40

Initial Commit.

73722db

Add truncator class

9a29659

Changes

e238a3f

Reduce allocations

6e757e8

Fixes and unit test suite.

2e1186b

Move previous CIAgentWriter logic to new OriginTagTraceProcessor.

e3853fa

Initial event model structure.

934e436

Initial writer work

1b5fb65

Changes

29e85a8

changes

302a2f9

Custom Meta enumerator for tagslist.

50ec294

Optimize TagsList enumerators.

439fc4c

Changes to use the AgentPayload format.

6fa05bc

Merge branch 'master' into tony/agentless-poc

e55b79a

CompactWhiteSpace method port.

4607896

ReplaceDigits method port.

df32238

Merge branch 'master' into tony/agentless-poc

060878c

Merge branch 'master' into tony/agentless-poc

5958b46

fix merge

ecfa9f1

Fix from merge

4d07844

add CI App MessagePack support

c087426

Formatter changes

6214dd8

Fix messagepack formatters

cb0e129

Revert source generator changes and json support.

29cde8b

Update the writer.

a15b84b

multiple file in the file writer.

f6cc592

Add SpanEvent message.

7c11c22

Merge branch 'master' into tony/agentless-poc

d1f8879

Merge branch 'master' into tony/agentless-poc

f166fb9

Merge branch 'master' into tony/agentless-poc

40beb0a

lucaspimentel reviewed Mar 1, 2022

View reviewed changes

Comment thread tracer/src/Datadog.Trace/TraceProcessors/TraceUtil.cs Outdated

lucaspimentel approved these changes Mar 1, 2022

View reviewed changes

This comment has been minimized.

Sign in to view

tonyredondo added 2 commits March 1, 2022 09:59

Remove #if DEBUG preprocessor with a runtime check.

90dbbc1

Move ArraySlice to Datadog.Trace.Util namespace

5c7da39

andrewlock approved these changes Mar 1, 2022

View reviewed changes

This comment has been minimized.

Sign in to view

tonyredondo and others added 12 commits March 1, 2022 11:59

Apply suggested optimization.

7b643b4

Update tracer/src/Datadog.Trace.Tools.Runner/LegacyCommand.cs

3b2ac80

Co-authored-by: Andrew Lock <[email protected]>

Update tracer/src/Datadog.Trace.Tools.Runner/RunCommand.cs

a036678

Co-authored-by: Andrew Lock <[email protected]>

Add DefaultMaxBufferSize const.

3fe7151

Merge branch 'tony/agentless-poc' of github.com:DataDog/dd-trace-dotn…

badb95d

…et into tony/agentless-poc

Use #nullable enable and remove some comments.

4cb8983

Remove some comments and add readonly to some fields.

52254ed

Update tracer/src/Datadog.Trace/Ci/CITracerManagerFactory.cs

2de4ac4

Co-authored-by: Andrew Lock <[email protected]>

remove comment.

5b2e33c

Merge branch 'tony/agentless-poc' of github.com:DataDog/dd-trace-dotn…

4014bd0

…et into tony/agentless-poc

Change the algorithm to find the root span.

1190e65

Rename methods.

b4d13f1

This comment has been minimized.

Sign in to view

fixes.

23f9667

This comment has been minimized.

Sign in to view

tonyredondo added 3 commits March 1, 2022 18:11

Processors refactor

63cb777

Change NormalizerTraceProcessorTests to avoid VSTest serializer bug.

644ad0a

Fix SourceGenerators tests

d61c70c

tonyredondo merged commit bfc8b54 into master Mar 1, 2022

tonyredondo deleted the tony/agentless-poc branch March 1, 2022 21:48

github-actions Bot added this to the vNext milestone Mar 1, 2022

Uh oh!

Conversation

tonyredondo commented Feb 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

lucaspimentel left a comment

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

andrewlock left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andrewlock Mar 1, 2022

Choose a reason for hiding this comment

Uh oh!

tonyredondo Mar 1, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andrewlock Mar 1, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andrewlock Mar 1, 2022

Choose a reason for hiding this comment

Uh oh!

tonyredondo Mar 1, 2022

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

andrewlock commented Mar 1, 2022

Benchmarks Report 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Uh oh!

andrewlock commented Mar 1, 2022

Code Coverage Report 📊

Datadog.Trace Breakdown ⚠️

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tonyredondo commented Feb 4, 2022 •

edited

Loading