Reorganize code around DBM injection to make sure comments get injected even if setting context fails#6167
Conversation
…ed even if setting context fails
Datadog ReportBranch report: ✅ 0 Failed, 365339 Passed, 2069 Skipped, 15h 3m 31.79s Total Time |
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6167) - mean (70ms) : 68, 73
. : milestone, 70,
master - mean (72ms) : 66, 78
. : milestone, 72,
section CallTarget+Inlining+NGEN
This PR (6167) - mean (1,113ms) : 1086, 1140
. : milestone, 1113,
master - mean (1,115ms) : 1091, 1139
. : milestone, 1115,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6167) - mean (110ms) : 106, 114
. : milestone, 110,
master - mean (110ms) : 108, 112
. : milestone, 110,
section CallTarget+Inlining+NGEN
This PR (6167) - mean (772ms) : 753, 790
. : milestone, 772,
master - mean (779ms) : 766, 792
. : milestone, 779,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6167) - mean (93ms) : 90, 96
. : milestone, 93,
master - mean (92ms) : 90, 95
. : milestone, 92,
section CallTarget+Inlining+NGEN
This PR (6167) - mean (726ms) : 710, 742
. : milestone, 726,
master - mean (729ms) : 711, 747
. : milestone, 729,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6167) - mean (190ms) : 187, 192
. : milestone, 190,
master - mean (190ms) : 186, 194
. : milestone, 190,
section CallTarget+Inlining+NGEN
This PR (6167) - mean (1,198ms) : 1173, 1223
. : milestone, 1198,
master - mean (1,202ms) : 1173, 1230
. : milestone, 1202,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6167) - mean (276ms) : 270, 282
. : milestone, 276,
master - mean (275ms) : 270, 280
. : milestone, 275,
section CallTarget+Inlining+NGEN
This PR (6167) - mean (939ms) : 924, 954
. : milestone, 939,
master - mean (947ms) : 927, 966
. : milestone, 947,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6167) - mean (265ms) : 260, 270
. : milestone, 265,
master - mean (264ms) : 261, 268
. : milestone, 264,
section CallTarget+Inlining+NGEN
This PR (6167) - mean (927ms) : 907, 946
. : milestone, 927,
master - mean (926ms) : 910, 942
. : milestone, 926,
|
Benchmarks Report for tracer 🐌Benchmarks for #6167 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉
|
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 | 41.89 KB | 41.66 KB | -225 B | -0.54% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 570μs | 2.35μs | 10.2μs | 0.592 | 0 | 0 | 41.54 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 676μs | 3.52μs | 17.6μs | 0.343 | 0 | 0 | 41.89 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 902μs | 3.84μs | 15.3μs | 8.09 | 2.25 | 0.45 | 53.28 KB |
| #6167 | WriteAndFlushEnrichedTraces |
net6.0 | 588μs | 3.05μs | 14μs | 0.558 | 0 | 0 | 41.61 KB |
| #6167 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 722μs | 3.51μs | 17.5μs | 0.349 | 0 | 0 | 41.66 KB |
| #6167 | WriteAndFlushEnrichedTraces |
net472 | 886μs | 3.34μs | 12.9μs | 8.19 | 2.59 | 0.431 | 53.31 KB |
Benchmarks.Trace.DbCommandBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #6167
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0
1.127
1,359.87
1,207.05
| Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0 | 1.127 | 1,359.87 | 1,207.05 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteNonQuery |
net6.0 | 1.36μs | 1.2ns | 4.49ns | 0.0144 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
netcoreapp3.1 | 1.74μs | 1.93ns | 7.48ns | 0.0138 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
net472 | 2.12μs | 2.51ns | 9.7ns | 0.156 | 0 | 0 | 987 B |
| #6167 | ExecuteNonQuery |
net6.0 | 1.21μs | 1.33ns | 5.14ns | 0.0139 | 0 | 0 | 1.02 KB |
| #6167 | ExecuteNonQuery |
netcoreapp3.1 | 1.76μs | 1.9ns | 7.38ns | 0.0139 | 0 | 0 | 1.02 KB |
| #6167 | ExecuteNonQuery |
net472 | 2.1μs | 2.07ns | 8.03ns | 0.156 | 0 | 0 | 987 B |
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | CallElasticsearch |
net6.0 | 1.15μs | 0.863ns | 3.34ns | 0.0138 | 0 | 0 | 976 B |
| master | CallElasticsearch |
netcoreapp3.1 | 1.57μs | 1.28ns | 4.77ns | 0.0134 | 0 | 0 | 976 B |
| master | CallElasticsearch |
net472 | 2.52μs | 4.34ns | 16.8ns | 0.158 | 0 | 0 | 995 B |
| master | CallElasticsearchAsync |
net6.0 | 1.26μs | 0.482ns | 1.8ns | 0.0134 | 0 | 0 | 952 B |
| master | CallElasticsearchAsync |
netcoreapp3.1 | 1.65μs | 1.69ns | 6.31ns | 0.0139 | 0 | 0 | 1.02 KB |
| master | CallElasticsearchAsync |
net472 | 2.66μs | 2.27ns | 8.81ns | 0.167 | 0 | 0 | 1.05 KB |
| #6167 | CallElasticsearch |
net6.0 | 1.16μs | 2.17ns | 8.39ns | 0.014 | 0 | 0 | 976 B |
| #6167 | CallElasticsearch |
netcoreapp3.1 | 1.61μs | 0.887ns | 3.44ns | 0.013 | 0 | 0 | 976 B |
| #6167 | CallElasticsearch |
net472 | 2.42μs | 0.791ns | 2.96ns | 0.158 | 0 | 0 | 995 B |
| #6167 | CallElasticsearchAsync |
net6.0 | 1.23μs | 0.485ns | 1.81ns | 0.0136 | 0 | 0 | 952 B |
| #6167 | CallElasticsearchAsync |
netcoreapp3.1 | 1.74μs | 0.924ns | 3.46ns | 0.014 | 0 | 0 | 1.02 KB |
| #6167 | CallElasticsearchAsync |
net472 | 2.64μs | 1.21ns | 4.51ns | 0.166 | 0 | 0 | 1.05 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteAsync |
net6.0 | 1.31μs | 0.729ns | 2.82ns | 0.0131 | 0 | 0 | 952 B |
| master | ExecuteAsync |
netcoreapp3.1 | 1.56μs | 0.847ns | 3.17ns | 0.0125 | 0 | 0 | 952 B |
| master | ExecuteAsync |
net472 | 1.83μs | 0.305ns | 1.1ns | 0.145 | 0 | 0 | 915 B |
| #6167 | ExecuteAsync |
net6.0 | 1.23μs | 1.04ns | 3.88ns | 0.0136 | 0 | 0 | 952 B |
| #6167 | ExecuteAsync |
netcoreapp3.1 | 1.58μs | 1.3ns | 5.02ns | 0.0129 | 0 | 0 | 952 B |
| #6167 | ExecuteAsync |
net472 | 1.75μs | 0.478ns | 1.79ns | 0.145 | 0 | 0 | 915 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendAsync |
net6.0 | 4.18μs | 2.73ns | 10.2ns | 0.0314 | 0 | 0 | 2.22 KB |
| master | SendAsync |
netcoreapp3.1 | 5.08μs | 1.25ns | 4.85ns | 0.0356 | 0 | 0 | 2.76 KB |
| master | SendAsync |
net472 | 7.84μs | 5.69ns | 22ns | 0.496 | 0 | 0 | 3.15 KB |
| #6167 | SendAsync |
net6.0 | 4.1μs | 1.51ns | 5.66ns | 0.031 | 0 | 0 | 2.22 KB |
| #6167 | SendAsync |
netcoreapp3.1 | 5.12μs | 2.79ns | 10.8ns | 0.036 | 0 | 0 | 2.76 KB |
| #6167 | SendAsync |
net472 | 7.81μs | 3.35ns | 13ns | 0.498 | 0 | 0 | 3.15 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 1.54μs | 1.1ns | 4.1ns | 0.0233 | 0 | 0 | 1.64 KB |
| master | EnrichedLog |
netcoreapp3.1 | 2.24μs | 1.1ns | 4.26ns | 0.0214 | 0 | 0 | 1.64 KB |
| master | EnrichedLog |
net472 | 2.66μs | 1.08ns | 4.19ns | 0.249 | 0 | 0 | 1.57 KB |
| #6167 | EnrichedLog |
net6.0 | 1.55μs | 1.24ns | 4.82ns | 0.0226 | 0 | 0 | 1.64 KB |
| #6167 | EnrichedLog |
netcoreapp3.1 | 2.18μs | 1.1ns | 4.1ns | 0.0218 | 0 | 0 | 1.64 KB |
| #6167 | EnrichedLog |
net472 | 2.51μs | 0.92ns | 3.44ns | 0.249 | 0 | 0 | 1.57 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 117μs | 148ns | 571ns | 0.058 | 0 | 0 | 4.28 KB |
| master | EnrichedLog |
netcoreapp3.1 | 123μs | 210ns | 813ns | 0 | 0 | 0 | 4.28 KB |
| master | EnrichedLog |
net472 | 153μs | 358ns | 1.38μs | 0.691 | 0.23 | 0 | 4.46 KB |
| #6167 | EnrichedLog |
net6.0 | 115μs | 114ns | 443ns | 0.0571 | 0 | 0 | 4.28 KB |
| #6167 | EnrichedLog |
netcoreapp3.1 | 121μs | 216ns | 835ns | 0 | 0 | 0 | 4.28 KB |
| #6167 | EnrichedLog |
net472 | 150μs | 143ns | 554ns | 0.68 | 0.227 | 0 | 4.46 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.98μs | 1.17ns | 4.54ns | 0.0313 | 0 | 0 | 2.2 KB |
| master | EnrichedLog |
netcoreapp3.1 | 3.97μs | 2.78ns | 10.8ns | 0.0279 | 0 | 0 | 2.2 KB |
| master | EnrichedLog |
net472 | 4.75μs | 1.73ns | 6.7ns | 0.32 | 0 | 0 | 2.02 KB |
| #6167 | EnrichedLog |
net6.0 | 3.12μs | 1.09ns | 4.21ns | 0.0306 | 0 | 0 | 2.2 KB |
| #6167 | EnrichedLog |
netcoreapp3.1 | 4.32μs | 9.02ns | 32.5ns | 0.03 | 0 | 0 | 2.2 KB |
| #6167 | EnrichedLog |
net472 | 4.79μs | 2.5ns | 9.7ns | 0.321 | 0 | 0 | 2.02 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendReceive |
net6.0 | 1.34μs | 0.851ns | 3.3ns | 0.0161 | 0 | 0 | 1.14 KB |
| master | SendReceive |
netcoreapp3.1 | 1.82μs | 0.6ns | 2.32ns | 0.0148 | 0 | 0 | 1.14 KB |
| master | SendReceive |
net472 | 2.08μs | 0.868ns | 3.36ns | 0.183 | 0.00103 | 0 | 1.16 KB |
| #6167 | SendReceive |
net6.0 | 1.4μs | 1.25ns | 4.68ns | 0.0161 | 0 | 0 | 1.14 KB |
| #6167 | SendReceive |
netcoreapp3.1 | 1.74μs | 1.26ns | 4.71ns | 0.0149 | 0 | 0 | 1.14 KB |
| #6167 | SendReceive |
net472 | 2.09μs | 1.54ns | 5.97ns | 0.184 | 0.00104 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.75μs | 0.663ns | 2.57ns | 0.0219 | 0 | 0 | 1.6 KB |
| master | EnrichedLog |
netcoreapp3.1 | 4.03μs | 1.39ns | 5.2ns | 0.0221 | 0 | 0 | 1.65 KB |
| master | EnrichedLog |
net472 | 4.33μs | 3.08ns | 11.9ns | 0.323 | 0 | 0 | 2.04 KB |
| #6167 | EnrichedLog |
net6.0 | 2.71μs | 1.24ns | 4.81ns | 0.0217 | 0 | 0 | 1.6 KB |
| #6167 | EnrichedLog |
netcoreapp3.1 | 3.92μs | 1.83ns | 6.34ns | 0.0216 | 0 | 0 | 1.65 KB |
| #6167 | EnrichedLog |
net472 | 4.37μs | 3.68ns | 13.8ns | 0.322 | 0 | 0 | 2.04 KB |
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️
Slower ⚠️ in #6167
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0
1.168
394.19
460.24
| Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 | 1.168 | 394.19 | 460.24 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 394ns | 0.357ns | 1.38ns | 0.00814 | 0 | 0 | 576 B |
| master | StartFinishSpan |
netcoreapp3.1 | 590ns | 0.504ns | 1.95ns | 0.008 | 0 | 0 | 576 B |
| master | StartFinishSpan |
net472 | 696ns | 1.01ns | 3.77ns | 0.0918 | 0 | 0 | 578 B |
| master | StartFinishScope |
net6.0 | 476ns | 0.215ns | 0.834ns | 0.00975 | 0 | 0 | 696 B |
| master | StartFinishScope |
netcoreapp3.1 | 674ns | 0.597ns | 2.31ns | 0.00941 | 0 | 0 | 696 B |
| master | StartFinishScope |
net472 | 836ns | 0.69ns | 2.67ns | 0.104 | 0 | 0 | 658 B |
| #6167 | StartFinishSpan |
net6.0 | 460ns | 0.327ns | 1.27ns | 0.00809 | 0 | 0 | 576 B |
| #6167 | StartFinishSpan |
netcoreapp3.1 | 634ns | 0.767ns | 2.97ns | 0.00771 | 0 | 0 | 576 B |
| #6167 | StartFinishSpan |
net472 | 753ns | 0.908ns | 3.52ns | 0.0916 | 0 | 0 | 578 B |
| #6167 | StartFinishScope |
net6.0 | 479ns | 0.349ns | 1.35ns | 0.00965 | 0 | 0 | 696 B |
| #6167 | StartFinishScope |
netcoreapp3.1 | 728ns | 1.53ns | 5.93ns | 0.00927 | 0 | 0 | 696 B |
| #6167 | StartFinishScope |
net472 | 866ns | 0.985ns | 3.82ns | 0.104 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 616ns | 0.22ns | 0.823ns | 0.00989 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 885ns | 0.675ns | 2.61ns | 0.00918 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
net472 | 1.14μs | 1.07ns | 4.13ns | 0.104 | 0 | 0 | 658 B |
| #6167 | RunOnMethodBegin |
net6.0 | 587ns | 0.465ns | 1.8ns | 0.00966 | 0 | 0 | 696 B |
| #6167 | RunOnMethodBegin |
netcoreapp3.1 | 904ns | 2.61ns | 10.1ns | 0.00929 | 0 | 0 | 696 B |
| #6167 | RunOnMethodBegin |
net472 | 1.17μs | 1ns | 3.89ns | 0.104 | 0 | 0 | 658 B |
Throughput/Crank Report ⚡Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6167) (11.158M) : 0, 11157548
master (11.147M) : 0, 11146507
benchmarks/2.9.0 (11.081M) : 0, 11080577
section Automatic
This PR (6167) (7.392M) : 0, 7392290
master (7.346M) : 0, 7346161
benchmarks/2.9.0 (7.732M) : 0, 7732233
section Trace stats
master (7.696M) : 0, 7695552
section Manual
master (11.000M) : 0, 11000138
section Manual + Automatic
This PR (6167) (6.809M) : 0, 6808630
master (6.847M) : 0, 6847370
section DD_TRACE_ENABLED=0
master (10.226M) : 0, 10225834
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6167) (9.540M) : 0, 9539763
master (9.607M) : 0, 9606697
benchmarks/2.9.0 (9.798M) : 0, 9798067
section Automatic
This PR (6167) (6.525M) : 0, 6525302
master (6.391M) : 0, 6390624
section Trace stats
master (6.768M) : 0, 6767573
section Manual
master (9.365M) : 0, 9364877
section Manual + Automatic
This PR (6167) (6.004M) : 0, 6004318
master (6.038M) : 0, 6038189
section DD_TRACE_ENABLED=0
master (8.878M) : 0, 8877751
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (6167) (10.043M) : 0, 10042808
master (10.185M) : 0, 10184870
benchmarks/2.9.0 (10.067M) : 0, 10067315
section Automatic
This PR (6167) (6.537M) : 0, 6537476
master (6.539M) : 0, 6539096
benchmarks/2.9.0 (7.552M) : 0, 7552193
section Trace stats
master (7.311M) : 0, 7310576
section Manual
master (9.936M) : 0, 9936096
section Manual + Automatic
This PR (6167) (6.252M) : 0, 6251672
master (6.250M) : 0, 6249990
section DD_TRACE_ENABLED=0
master (9.355M) : 0, 9355348
|
bouwkast
left a comment
There was a problem hiding this comment.
Looks good to me, should help us root cause the issue
Currently, if the
set context_infofails for some reason, there is no injection applied. We can make sure activating full mode doesn't cause any regression.Also moved the debug log before sending the query to the DB so that we can see what's happening in the debug logs if it fails.