[Perf] Windows/x64: 42 Improvements on 3/15/2023 12:03:45 AM #14277
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | a78f9bde2edd384f5d26c9d394eb7fcc5f48a664 |
| Compare | 1cfda5e9a887d39004644c2f96a87b31fcca0ac7 |
| Diff | Diff |
| Configs | CompliationMode:tiered, RunKind:micro |
Improvements in System.Memory.Slice<String>
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Slice<String>*'Details
Payloads
Histogram
System.Memory.Slice<String>.SpanStart
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.93645499850241 < 3.9400451742199345.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 68.10935867975684 (T) = (0 -2.9522621426140865) / Math.Sqrt((0.192197693199618 / (299)) + (4.2716814901605324E-05 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.36929940162282116 = (4.680924911456229 - 2.9522621426140865) / 4.680924911456229 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.ReadOnlyMemoryStartLengthSpan
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.617986263395465 < 9.41932227942982.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/24/2023 9:01:57 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 43.89021020299283 (T) = (0 -7.644181500282852) / Math.Sqrt((0.29187250730843967 / (299)) + (0.00042143088643114955 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.15299787904370352 = (9.02498507518764 - 7.644181500282852) / 9.02498507518764 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.MemoryStartSpan
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.227990953065581 < 8.862555234804866.
IsChangePoint: Marked as a change because one of 2/20/2023 10:31:48 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 113.35773674716036 (T) = (0 -7.253173766266214) / Math.Sqrt((0.11354785065723538 / (299)) + (0.0030266168032469123 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.25455255257347237 = (9.729959893626301 - 7.253173766266214) / 9.729959893626301 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.ReadOnlyMemorySpanStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.522934138740015 < 7.231113771002739.
IsChangePoint: Marked as a change because one of 1/5/2023 11:35:29 PM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 48.69748226364586 (T) = (0 -6.546967416815355) / Math.Sqrt((0.13152497094428625 / (299)) + (0.00035188701032371633 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.13644490045576002 = (7.581412489221197 - 6.546967416815355) / 7.581412489221197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.ReadOnlyMemoryStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.518851367579754 < 5.500211285585415.
IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 69.7082573714671 (T) = (0 -4.559742227207591) / Math.Sqrt((0.13284661393268782 / (299)) + (0.0019704379924729127 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.25624184897441343 = (6.130678663380092 - 4.559742227207591) / 6.130678663380092 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.MemorySpanStart
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.397997100549765 < 7.60462308589293.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 93.5630659795562 (T) = (0 -6.4203258564365) / Math.Sqrt((0.049594569483417915 / (299)) + (0.00027044365740467186 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.16146577435378404 = (7.656605610210697 - 6.4203258564365) / 7.656605610210697 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.ReadOnlySpanStart
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.8110755856433087 < 3.662434662075134.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 141.3155359694692 (T) = (0 -2.813741651800149) / Math.Sqrt((0.022083721145659645 / (299)) + (2.9395386241067333E-05 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.3028378422918251 = (4.035993090976049 - 2.813741651800149) / 4.035993090976049 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.MemoryStart
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.335439033905408 < 5.805991982967645.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 71.13519531887525 (T) = (0 -4.38554930107501) / Math.Sqrt((0.07362562819203194 / (299)) + (0.014743886935018017 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.3035224293906388 = (6.29675597053041 - 4.38554930107501) / 6.29675597053041 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.MemorySpanStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.680069120976609 < 8.219165365039117.
IsChangePoint: Marked as a change because one of 1/5/2023 11:35:29 PM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 45.03443334295466 (T) = (0 -6.700011780985875) / Math.Sqrt((0.18278784898321165 / (299)) + (0.00020177816065013144 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.14315577444379393 = (7.819404719261164 - 6.700011780985875) / 7.819404719261164 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.MemoryStartLengthSpan
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.768177151917542 < 9.605286041152112.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/24/2023 9:01:57 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 46.892124525688125 (T) = (0 -7.780982910464876) / Math.Sqrt((0.2958812690361042 / (299)) + (0.0008103421019252241 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.16112018636982722 = (9.275444210289685 - 7.780982910464876) / 9.275444210289685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.ReadOnlyMemoryStartSpan
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.062465429531682 < 7.733234225090228.
IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 69.76210008981636 (T) = (0 -7.103414509039762) / Math.Sqrt((0.22475803122050161 / (299)) + (0.0008197857953980693 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.21504411799563047 = (9.049444270551017 - 7.103414509039762) / 9.049444270551017 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.SpanStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4771403096717033 < 4.426270200245299.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 149.34748364345708 (T) = (0 -3.491643135479506) / Math.Sqrt((0.014277160619702646 / (299)) + (4.181453965441008E-05 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.23059808134874327 = (4.538126369115733 - 3.491643135479506) / 4.538126369115733 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.MemoryStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.679867180623703 < 6.478120588210451.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 2/9/2023 12:40:40 PM, 2/24/2023 9:01:57 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 59.84779875429452 (T) = (0 -4.753722809325139) / Math.Sqrt((0.2360008950402861 / (299)) + (0.0053928812676165 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.28095798841957503 = (6.611189238966233 - 4.753722809325139) / 6.611189238966233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Slice<String>.ReadOnlyMemoryStart
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.12561742615315 < 5.116817857301745.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 60.083780674347395 (T) = (0 -4.128607102443564) / Math.Sqrt((0.05620245327960974 / (299)) + (0.016362587307529236 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.2826979057384632 = (5.755743828817298 - 4.128607102443564) / 5.755743828817298 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | a78f9bde2edd384f5d26c9d394eb7fcc5f48a664 |
| Compare | 1cfda5e9a887d39004644c2f96a87b31fcca0ac7 |
| Diff | Diff |
| Configs | CompliationMode:tiered, RunKind:micro |
Improvements in System.Memory.Constructors<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ArrayAsMemoryStartLength - Duration of single invocation | 10.48 ns | 7.55 ns | 0.72 | 0.11 | False | |||||
| SpanImplicitCastFromArraySegment - Duration of single invocation | 7.80 ns | 4.17 ns | 0.53 | 0.03 | False | |||||
| SpanImplicitCastFromArray - Duration of single invocation | 5.36 ns | 3.11 ns | 0.58 | 0.07 | False | |||||
| ArrayAsSpan - Duration of single invocation | 7.61 ns | 4.99 ns | 0.66 | 0.06 | False | |||||
| MemoryFromArray - Duration of single invocation | 7.54 ns | 5.31 ns | 0.70 | 0.10 | False | |||||
| ArrayAsSpanStartLength - Duration of single invocation | 9.46 ns | 6.71 ns | 0.71 | 0.04 | False | |||||
| MemoryFromArrayStartLength - Duration of single invocation | 8.37 ns | 5.55 ns | 0.66 | 0.16 | False | |||||
| SpanFromArrayStartLength - Duration of single invocation | 7.07 ns | 4.30 ns | 0.61 | 0.06 | False | |||||
| ArrayAsMemory - Duration of single invocation | 8.83 ns | 5.99 ns | 0.68 | 0.12 | False | |||||
| SpanFromArray - Duration of single invocation | 6.27 ns | 3.67 ns | 0.59 | 0.06 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors<String>*'Details
Payloads
Histogram
System.Memory.Constructors<String>.ArrayAsMemoryStartLength
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.554994088056087 < 9.969406213633361.
IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 2/9/2023 12:40:40 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 40.5257815609522 (T) = (0 -7.628961558758105) / Math.Sqrt((0.3119559740716691 / (299)) + (0.1669213748196305 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.29868497889110546 = (10.878080932439548 - 7.628961558758105) / 10.878080932439548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanImplicitCastFromArraySegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.1745008983446015 < 7.410923512608309.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 107.12686379244143 (T) = (0 -4.188995476336384) / Math.Sqrt((0.303703995977631 / (299)) + (0.00022535878155941839 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.4499298650096105 = (7.615384311692493 - 4.188995476336384) / 7.615384311692493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanImplicitCastFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.10638370256486 < 5.054340955589267.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 115.63364752972849 (T) = (0 -3.1048169073754717) / Math.Sqrt((0.08240567507142349 / (299)) + (4.624638158636564E-06 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.3821271075591124 = (5.025009100350704 - 3.1048169073754717) / 5.025009100350704 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsSpan
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.989732968952912 < 7.227380525179633.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 134.14245601706654 (T) = (0 -4.950451975904407) / Math.Sqrt((0.09571731195063066 / (299)) + (0.0007813043488323535 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.33490471667823957 = (7.443222196945686 - 4.950451975904407) / 7.443222196945686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.MemoryFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.310605811104224 < 7.159521786421577.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 118.24444299943531 (T) = (0 -5.324660612066489) / Math.Sqrt((0.14657355003777936 / (299)) + (0.0015291528816950505 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.34029234388060614 = (8.07124271285224 - 5.324660612066489) / 8.07124271285224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsSpanStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.705621125980745 < 8.986341555713459.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 144.11234239009852 (T) = (0 -6.586845533854212) / Math.Sqrt((0.05873482637326422 / (299)) + (0.004875656636730411 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.29152512520303553 = (9.297218247495195 - 6.586845533854212) / 9.297218247495195 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.MemoryFromArrayStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.55478755237783 < 7.955321900605186.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 58.17700493835139 (T) = (0 -5.83640985644542) / Math.Sqrt((0.15390873609340594 / (299)) + (0.04633278777146588 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.30882944855659944 = (8.444239767242687 - 5.83640985644542) / 8.444239767242687 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromArrayStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.298763104061341 < 6.712404263667414.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 167.4223470295279 (T) = (0 -4.325948354451741) / Math.Sqrt((0.06228267615400356 / (299)) + (0.00037570509218120326 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.36491525441499323 = (6.811608032667993 - 4.325948354451741) / 6.811608032667993 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsMemory
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.985955618880904 < 8.257438171898773.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 39.167243548712314 (T) = (0 -6.122929033142202) / Math.Sqrt((0.1864585459340132 / (299)) + (0.1641264105428004 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.32980270306622844 = (9.136009740945383 - 6.122929033142202) / 9.136009740945383 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.6722859473684446 < 5.962390017158344.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 173.6180832738909 (T) = (0 -3.6720593177553416) / Math.Sqrt((0.05598591964050953 / (299)) + (8.187757655228768E-06 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.3929952268860507 = (6.049473546834875 - 3.6720593177553416) / 6.049473546834875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | a78f9bde2edd384f5d26c9d394eb7fcc5f48a664 |
| Compare | 1cfda5e9a887d39004644c2f96a87b31fcca0ac7 |
| Diff | Diff |
| Configs | CompliationMode:tiered, RunKind:micro |
Improvements in System.Collections.IndexerSetReverse<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Span - Duration of single invocation | 1.07 μs | 224.29 ns | 0.21 | 0.06 | False |
_1.png)
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse<String>*'Details
Payloads
Histogram
System.Collections.IndexerSetReverse<String>.Span(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 224.287289961851 < 1.0084892631494153.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM.
IsImprovementStdDev: Marked as improvement because 10.49507291162359 (T) = (0 -223.81201969847356) / Math.Sqrt((145540.45888220915 / (299)) + (0.22192239491125595 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.5084965977556369 = (455.36209653173444 - 223.81201969847356) / 455.36209653173444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository