Conversation
| /// </summary> | ||
| public class Perf_Regex | ||
| { | ||
| private const int innerIterations = 100; |
There was a problem hiding this comment.
Chosen to get ~250ms outer iteration.
| } | ||
| } | ||
|
|
||
| // A series of patterns (all valid and non pathological) and inputs (which they may or may not match) |
There was a problem hiding this comment.
Actually all of these do match. I'll leave the comment, because it would be nice to add more that don't.
|
@dotnet-bot test NETFX x86 Release Build |
|
@dotnet-bot test OSX x64 Debug Build |
| { | ||
| for (int i = 0; i < innerIterations; i++) | ||
| { | ||
| foreach(var test in Match_TestData()) |
There was a problem hiding this comment.
There's work here unrelated to Regex.Match that it would be nice to move out of the measurement, e.g. store the data from Match_TestData() into an array at the beginning of the method once and then access the array here.
| } | ||
|
|
||
| // A series of patterns (all valid and non pathological) and inputs (which they may or may not match) | ||
| public static IEnumerable<object[]> Match_TestData() |
There was a problem hiding this comment.
Are you planning to make this a Theory? If not we don't need to be constrained to use object[] and can use a type more easily accessed, like a value tuple with names.
There was a problem hiding this comment.
Not a theory, since each would have too low duration.
This would make sense, but does not seem essential now, I will follow up with cleanup next time I work on regex perf.
|
@dotnet/dnceng Windows and OSX queues are timing out, Is this a helix thing? |
|
@dotnet/dnceng any thoughts? |
|
@danmosemsft I'll take a quick look and chat w/ the FR team in their standup in 5 mins. |
|
Thanks @MattGal ! |
|
Filed https://github.com/dotnet/core-eng/issues/2575. I suspect this relates to recent issues w/ EventHub we've been seeing but I can assure you those jobs ran and tests executed, so we should also try to harden against problems like this. @Chrisboh FYI. |
ViktorHofer
left a comment
ViktorHofer
left a comment
There was a problem hiding this comment.
LGTM. I hesitated to add benchmark tests for Regex as I wanted to see if we could use BenchmarkDotNet somehow in our innerloop. But I guess we can easily switch later on.
|
@MattGal should I run again? Or can you see that tests all passed? |
|
@danmosemsft sorry for the slow reply; for the run I checked everything passed but I'd suggest rerunning either way, I don't want to be responsible for manually aggregating a bunch of runs :) |
|
No problem |
|
As discussed yesterday it would be great if we could add a benchmark with Invariant mode enabled |
* Regex perf test * caps Commit migrated from dotnet/corefx@80ef82c
5 participants
A crude start at regex perf tests.
The patterns and inputs are a subset of those in RegexGroupTests.Groups_Basic_TestData -- I excluded a small number that were unusually slow. All patterns are valid patterns.
I initially started by measuring each pattern separately but this produces too much data (and since each should take 100ms-1sec to measure, takes a long time). Rather than cherry pick a few of the patterns, I chose to start the tests with a single test that gets broad coverage, until we break tests out further.
Execution time is ~15 sec and produces a table like this, with matching CSV.