Skip to content

Exclude named pipe timeout from build log check#3157

Merged
andrewlock merged 1 commit into
masterfrom
andrew/exclude-timeouts
Sep 5, 2022
Merged

Exclude named pipe timeout from build log check#3157
andrewlock merged 1 commit into
masterfrom
andrew/exclude-timeouts

Conversation

@andrewlock

@andrewlock andrewlock commented Sep 2, 2022

Copy link
Copy Markdown
Member

Summary of changes

Exclude "The operation has timed out" errors from named pipes tests

Reason for change

We occasionally see them in the logs, even though the tests pass, causing flakiness. We think this is down to the mock named pipes server implementation so aren't currently investigating it further.

Implementation details

Exclude the phrase from the build log check

Test coverage

Did a manual test locally 👍

Other details

There's doesn't deal with the flaky debugger tests

@andrewlock andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Sep 2, 2022
@andrewlock
andrewlock requested review from a team as code owners September 2, 2022 14:33
@andrewlock

andrewlock commented Sep 2, 2022

Copy link
Copy Markdown
Member Author

Benchmarks Report 🐌

Benchmarks for #3157 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 706μs 1.18μs 4.56μs 0.355 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 493μs 681ns 2.55μs 0 0 0 2.58 KB
#3157 WriteAndFlushEnrichedTraces net472 706μs 857ns 3.32μs 0.355 0 0 3.18 KB
#3157 WriteAndFlushEnrichedTraces netcoreapp3.1 474μs 490ns 1.9μs 0 0 0 2.59 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 184ns 0.233ns 0.841ns 0.0675 9.25E-05 0 425 B
master AllCycleSimpleBody netcoreapp3.1 239ns 0.298ns 1.15ns 0.00577 0 0 424 B
master AllCycleMoreComplexBody net472 184ns 0.198ns 0.741ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 234ns 0.285ns 1.1ns 0.00562 0 0 400 B
master BodyExtractorSimpleBody net472 265ns 0.249ns 0.963ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 221ns 0.243ns 0.877ns 0.00366 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.2μs 12.2ns 45.5ns 1.21 0.0212 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.3μs 6.75ns 23.4ns 0.0923 0 0 6.75 KB
#3157 AllCycleSimpleBody net472 205ns 0.329ns 1.23ns 0.0675 0 0 425 B
#3157 AllCycleSimpleBody netcoreapp3.1 238ns 0.25ns 0.935ns 0.00585 0 0 424 B
#3157 AllCycleMoreComplexBody net472 185ns 0.141ns 0.528ns 0.0637 0 0 401 B
#3157 AllCycleMoreComplexBody netcoreapp3.1 236ns 0.261ns 1.01ns 0.00546 0 0 400 B
#3157 BodyExtractorSimpleBody net472 255ns 0.201ns 0.778ns 0.0573 0 0 361 B
#3157 BodyExtractorSimpleBody netcoreapp3.1 219ns 0.206ns 0.769ns 0.00375 0 0 272 B
#3157 BodyExtractorMoreComplexBody net472 14.4μs 6.36ns 22.9ns 1.21 0.0216 0 7.62 KB
#3157 BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 18.4ns 66.4ns 0.0909 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 183μs 126ns 489ns 0.273 0 0 20.57 KB
#3157 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3157 SendRequest netcoreapp3.1 184μs 306ns 1.19μs 0.182 0 0 20.57 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.81μs 3.61ns 14ns 0.15 0.000889 0 947 B
master ExecuteNonQuery netcoreapp3.1 1.51μs 1.67ns 6.48ns 0.0128 0 0 936 B
#3157 ExecuteNonQuery net472 1.86μs 2.83ns 10.9ns 0.15 0.000932 0 947 B
#3157 ExecuteNonQuery netcoreapp3.1 1.48μs 1.85ns 7.16ns 0.0119 0 0 936 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.54μs 1.43ns 5.53ns 0.183 0 0 1.16 KB
master CallElasticsearch netcoreapp3.1 1.51μs 0.615ns 2.3ns 0.0153 0 0 1.1 KB
master CallElasticsearchAsync net472 2.73μs 0.493ns 1.91ns 0.205 0 0 1.29 KB
master CallElasticsearchAsync netcoreapp3.1 1.62μs 0.828ns 3.1ns 0.0162 0 0 1.22 KB
#3157 CallElasticsearch net472 2.41μs 0.711ns 2.56ns 0.183 0 0 1.16 KB
#3157 CallElasticsearch netcoreapp3.1 1.48μs 0.675ns 2.34ns 0.0148 0 0 1.1 KB
#3157 CallElasticsearchAsync net472 2.64μs 1.91ns 7.4ns 0.204 0 0 1.29 KB
#3157 CallElasticsearchAsync netcoreapp3.1 1.62μs 0.439ns 1.64ns 0.0161 0 0 1.22 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.54μs 4.45ns 16.7ns 0.224 0 0 1.41 KB
master ExecuteAsync netcoreapp3.1 1.72μs 3.89ns 14.5ns 0.0179 0 0 1.34 KB
#3157 ExecuteAsync net472 2.6μs 6.59ns 25.5ns 0.224 0 0 1.41 KB
#3157 ExecuteAsync netcoreapp3.1 1.68μs 4.51ns 16.9ns 0.0183 0 0 1.34 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.76μs 14.2ns 54.9ns 0.438 0 0 2.77 KB
master SendAsync netcoreapp3.1 3.62μs 8.85ns 34.3ns 0.0343 0 0 2.6 KB
#3157 SendAsync net472 5.59μs 11.3ns 43.9ns 0.438 0 0 2.77 KB
#3157 SendAsync netcoreapp3.1 3.68μs 8.18ns 31.7ns 0.0346 0 0 2.6 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.06μs 2.98ns 11.5ns 0.287 0 0 1.81 KB
master EnrichedLog netcoreapp3.1 2.47μs 1.59ns 5.96ns 0.0246 0 0 1.85 KB
#3157 EnrichedLog net472 3.09μs 3.85ns 14.9ns 0.287 0 0 1.81 KB
#3157 EnrichedLog netcoreapp3.1 2.49μs 1.86ns 7.2ns 0.0247 0 0 1.85 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 152μs 88.3ns 342ns 0.683 0.228 0 4.65 KB
master EnrichedLog netcoreapp3.1 116μs 138ns 533ns 0 0 0 4.49 KB
#3157 EnrichedLog net472 153μs 132ns 510ns 0.695 0.232 0 4.65 KB
#3157 EnrichedLog netcoreapp3.1 117μs 183ns 709ns 0.0598 0 0 4.49 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.66μs 9.49ns 36.7ns 0.568 0.00284 0 3.59 KB
master EnrichedLog netcoreapp3.1 4.39μs 7.56ns 28.3ns 0.0547 0 0 3.91 KB
#3157 EnrichedLog net472 5.62μs 14.9ns 57.6ns 0.568 0.00276 0 3.59 KB
#3157 EnrichedLog netcoreapp3.1 4.45μs 9.84ns 36.8ns 0.0532 0 0 3.91 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.22μs 1.12ns 4.18ns 0.218 0 0 1.37 KB
master SendReceive netcoreapp3.1 1.88μs 7.85ns 30.4ns 0.0176 0 0 1.32 KB
#3157 SendReceive net472 2.42μs 2.04ns 7.63ns 0.218 0 0 1.37 KB
#3157 SendReceive netcoreapp3.1 1.88μs 0.995ns 3.86ns 0.0178 0 0 1.32 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.06μs 0.834ns 3.12ns 0.352 0 0 2.23 KB
master EnrichedLog netcoreapp3.1 4.22μs 0.964ns 3.73ns 0.0233 0 0 1.8 KB
#3157 EnrichedLog net472 5.19μs 2.93ns 11ns 0.354 0 0 2.23 KB
#3157 EnrichedLog netcoreapp3.1 4.27μs 3.79ns 14.7ns 0.0234 0 0 1.8 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.18μs 0.447ns 1.67ns 0.129 0 0 810 B
master StartFinishSpan netcoreapp3.1 1.01μs 0.213ns 0.767ns 0.0101 0 0 760 B
master StartFinishScope net472 1.37μs 0.645ns 2.41ns 0.141 0 0 891 B
master StartFinishScope netcoreapp3.1 1.09μs 0.316ns 1.14ns 0.0116 0 0 880 B
#3157 StartFinishSpan net472 1.16μs 0.946ns 3.66ns 0.129 0 0 810 B
#3157 StartFinishSpan netcoreapp3.1 938ns 0.274ns 1.06ns 0.0102 0 0 760 B
#3157 StartFinishScope net472 1.35μs 0.453ns 1.76ns 0.141 0 0 891 B
#3157 StartFinishScope netcoreapp3.1 1.06μs 0.528ns 1.98ns 0.0117 0 0 880 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.51μs 0.623ns 2.41ns 0.141 0 0 891 B
master RunOnMethodBegin netcoreapp3.1 1.25μs 0.423ns 1.58ns 0.0119 0 0 880 B
#3157 RunOnMethodBegin net472 1.54μs 1.36ns 5.28ns 0.141 0 0 891 B
#3157 RunOnMethodBegin netcoreapp3.1 1.21μs 0.639ns 2.39ns 0.0116 0 0 880 B

@andrewlock

andrewlock commented Sep 2, 2022

Copy link
Copy Markdown
Member Author

Code Coverage Report 📊

✔️ Merging #3157 into master will not change line coverage
✔️ Merging #3157 into master will not change branch coverage
⛔ Merging #3157 into master will will increase complexity by 5

master #3157 Change
Lines 17462 / 24055 17504 / 24063
Lines % 73% 73% 0% ✔️
Branches 10318 / 14797 10349 / 14803
Branches % 70% 70% 0% ✔️
Complexity 15977 15982 5 ⛔

View the full report for further details:

Datadog.Trace Breakdown ✔️

master #3157 Change
Lines % 73% 73% 0% ✔️
Branches % 70% 70% 0% ✔️
Complexity 15977 15982 5 ⛔

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Agent.Transports.SocketHandlerRequestFactory -7% ⛔ 0% ✔️ 0 ✔️
Datadog.Trace.Ci.CIVisibility 5% ✔️ 6% ✔️ 0 ✔️

View the full reports for further details:

@andrewlock
andrewlock merged commit e38945a into master Sep 5, 2022
@andrewlock
andrewlock deleted the andrew/exclude-timeouts branch September 5, 2022 09:06
@github-actions github-actions Bot added this to the vNext milestone Sep 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@robertpi robertpi robertpi approved these changes

Assignees

No one assigned

Labels

area:builds project files, build scripts, pipelines, versioning, releases, packages

Projects

None yet

Milestone

2.15.0

Development

Successfully merging this pull request may close these issues.

2 participants

@andrewlock @robertpi