[Params] should not change the order of provided values #660
Description
When I specify [Params] in some specific order, BenchmarkDotNet ignores this order and instead uses alphabetic order. I don't think that's user-friendly default behavior: if I specify the values in some order, BDN should respect it.
As an example, consider:
[Params(Ordinal, InvariantCulture, CurrentCulture, OrdinalIgnoreCase, InvariantCultureIgnoreCase, CurrentCultureIgnoreCase)]
public StringComparison StringComparison { get; set; }With this, the order of execution and the order in which the values are printed is alphabetic, not the one I provided:
// Found benchmarks:
// Program.Benchmark: DefaultJob [StringComparison=CurrentCulture]
// Program.Benchmark: DefaultJob [StringComparison=CurrentCultureIgnoreCase]
// Program.Benchmark: DefaultJob [StringComparison=InvariantCulture]
// Program.Benchmark: DefaultJob [StringComparison=InvariantCultureIgnoreCase]
// Program.Benchmark: DefaultJob [StringComparison=Ordinal]
// Program.Benchmark: DefaultJob [StringComparison=OrdinalIgnoreCase]
…
Method | StringComparison | Mean | Error | StdDev | Median |
---------- |--------------------------- |-----------:|----------:|------------:|-----------:|
Benchmark | CurrentCulture | 2,593.3 ns | 404.50 ns | 1,192.68 ns | 1,832.4 ns |
Benchmark | CurrentCultureIgnoreCase | 2,536.2 ns | 360.77 ns | 1,063.73 ns | 1,896.6 ns |
Benchmark | InvariantCulture | 1,154.3 ns | 67.55 ns | 176.76 ns | 1,180.4 ns |
Benchmark | InvariantCultureIgnoreCase | 984.7 ns | 31.38 ns | 92.52 ns | 972.2 ns |
Benchmark | Ordinal | 606.2 ns | 18.84 ns | 53.74 ns | 598.9 ns |
Benchmark | OrdinalIgnoreCase | 739.7 ns | 23.55 ns | 69.44 ns | 727.2 ns |
As far as I can tell, this is intentional behavior caused by this line:
But I don't think it's the right behavior: BDN should not change the order of values I provided, at least not for enums and string.