[Perf] Linux/x64: 4 Improvements on 2/19/2023 10:06:54 AM #13152
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | edce5ea1600bf4c61f10bc9de80e4a11ff14c842 |
| Compare | 7908e8ef62c7d5ebdcd6459faccd755b9b10d369 |
| Diff | Diff |
Improvements in System.Text.Json.Tests.Perf_Guids
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteGuids - Duration of single invocation | 3.85 ms | 2.22 ms | 0.58 | 0.02 | False | |||||
| WriteGuids - Duration of single invocation | 3.72 ms | 2.14 ms | 0.58 | 0.02 | False | |||||
| WriteGuids - Duration of single invocation | 3.66 ms | 2.06 ms | 0.56 | 0.03 | False | |||||
| WriteGuids - Duration of single invocation | 3.90 ms | 2.31 ms | 0.59 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Guids*'Details
Payloads
Histogram
System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: True)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2235298346354164 < 3.647053985791552.
IsChangePoint: Marked as a change because one of 2/19/2023 8:14:07 AM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsImprovementStdDev: Marked as improvement because 177.57262413165097 (T) = (0 -2232830.256026377) / Math.Sqrt((1335214915.3271708 / (299)) + (754894934.7266927 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.4155877218126646 = (3820642.2749226284 - 2232830.256026377) / 3820642.2749226284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: False)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.141704084735577 < 3.532486502628205.
IsChangePoint: Marked as a change because one of 2/19/2023 8:14:07 AM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsImprovementStdDev: Marked as improvement because 365.8757409974281 (T) = (0 -2142980.3134114584) / Math.Sqrt((5523524179.872078 / (299)) + (12912672.207961448 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.4315051016660328 = (3769568.2400874356 - 2142980.3134114584) / 3769568.2400874356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: True)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.055393794270833 < 3.480255973240327.
IsChangePoint: Marked as a change because one of 2/19/2023 8:14:07 AM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsImprovementStdDev: Marked as improvement because 360.981438754047 (T) = (0 -2058738.1348612062) / Math.Sqrt((3978481817.1641893 / (299)) + (66005775.97409806 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.4389319287642675 = (3669319.714320775 - 2058738.1348612062) / 3669319.714320775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: False)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3115700593112245 < 3.696352472994291.
IsChangePoint: Marked as a change because one of 2/19/2023 8:14:07 AM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsImprovementStdDev: Marked as improvement because 208.14934311390996 (T) = (0 -2308307.8581464225) / Math.Sqrt((5589030424.949237 / (299)) + (397205634.0714077 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.4079986168904219 = (3899159.5695633027 - 2308307.8581464225) / 3899159.5695633027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository