Currently BenchmarkDotNet's DefaultOrderer don't have parameter to customize Job display orders.
And JobComparer.Instance is always used.
Currently JobComparer.Instance sort jobs based on characteristics.
And compare each characteristics with Ordinal string comparer.
I want to add option to specify JobOrderPolicy that change behavior of JobComparer with custom NumericStringComparer.
(that similar to CompareOptions.NumericOrdering that added to .NET 10)
Background
Currently there is following job display order issues when using DefaultOrderer.
Example. 1: .NET Runtime orders
.NET 10.0 benchmark is shown on top item.
| Method |
Runtime |
Mean |
Error |
Ratio |
| Benchmark1 |
.NET 10.0 |
16.337 ms |
NA |
3.32 |
| Benchmark1 |
.NET 8.0 |
4.922 ms |
NA |
1.00 |
| Benchmark1 |
.NET 9.0 |
16.233 ms |
NA |
3.30 |
Example.2: JobId column orders (and NuGetReferences column orders)
Similarly, JobId/NuGetReferences are sorted by ordinal order.
| Type |
Method |
Job |
N |
Mean |
| ContainsBenchmark |
Contains |
v1.4.1 |
1000000 |
29.87 us |
| ContainsBenchmark |
Contains |
v1.4.10 |
1000000 |
29.90 us |
| ContainsBenchmark |
Contains |
v1.4.2 |
1000000 |
30.31 us |
Currently BenchmarkDotNet's DefaultOrderer don't have parameter to customize Job display orders.
And
JobComparer.Instanceis always used.Currently JobComparer.Instance sort jobs based on characteristics.
And compare each characteristics with
Ordinalstring comparer.I want to add option to specify
JobOrderPolicythat change behavior of JobComparer with customNumericStringComparer.(that similar to CompareOptions.NumericOrdering that added to .NET 10)
Background
Currently there is following job display order issues when using DefaultOrderer.
Example. 1: .NET
Runtimeorders.NET 10.0benchmark is shown on top item.Example.2:
JobIdcolumn orders (andNuGetReferencescolumn orders)Similarly, JobId/NuGetReferences are sorted by ordinal order.