🤖 Update Benchmark Results#6320
Conversation
thomhurst
added
automated
benchmarks
documentation
GitHub Actions
Failure
Up to standards ✅🟢 Issues
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Benchmark Update Review
This PR is a clean, automated documentation update — no code logic changes, only benchmark data and generated markdown. The changes look correct and consistent.
What's here
- Version bump: 1.56.18 → 1.56.35 across all benchmark tables
- Date update: 2026-06-21 → 2026-06-28
- All JSON source files, markdown docs, historical trends, and summary updated consistently
Performance snapshot (week-over-week)
| Scenario | TUnit | vs last week |
|---|---|---|
| AsyncTests | 351.8ms | ✅ -5% (improved) |
| DataDrivenTests | 282.91ms | |
| MassiveParallelTests | 462.5ms vs MSTest 2,829ms | ✅ Dominates |
| MatrixTests | 423.2ms vs NUnit 1,487ms | ✅ Strong lead |
| SetupTeardownTests | 337.0ms vs NUnit 1,002ms | ✅ ~3x faster |
| BuildTime | 953.1ms | ✅ Within noise |
One observation
In SetupTeardownTests, shows with in the chart. This is visually a bit misleading — a value of 0 in a bar chart could be read as "zero time" rather than "not applicable". This is likely a known limitation (AOT setup/teardown may not be benchmarked), but worth noting in case it catches users off guard when reading the docs.
Otherwise the update is correct, the data is consistent across JSON and markdown, and the automation is working as designed. No issues.
There was a problem hiding this comment.
Automated Benchmark Update Review
This PR is a clean, automated documentation update — no code logic changes, only benchmark data and generated markdown. The changes look correct and consistent.
What's here
- Version bump: 1.56.18 → 1.56.35 across all benchmark tables
- Date update: 2026-06-21 → 2026-06-28
- All JSON source files, markdown docs, historical trends, and summary updated consistently
Performance snapshot (week-over-week)
| Scenario | TUnit | vs last week |
|---|---|---|
| AsyncTests | 351.8ms | Improved ~5% |
| DataDrivenTests | 282.91ms | Slight regression ~3%, within normal variance |
| MassiveParallelTests | 462.5ms vs MSTest 2,829ms | Strong lead maintained |
| MatrixTests | 423.2ms vs NUnit 1,487ms | ~3.5x faster |
| SetupTeardownTests | 337.0ms vs NUnit 1,002ms | ~3x faster |
| BuildTime | 953.1ms | Within noise of other frameworks |
One observation
In SetupTeardownTests, the TUnit AOT result shows "NA" with a bar value of 0 in the chart. A bar of 0 could be read as "zero time" rather than "not applicable". This is likely a known limitation (AOT setup/teardown may not be benchmarked in this scenario), but it may be worth considering whether the chart should omit that bar entirely, or include a note in the docs clarifying that NA means "not measured" rather than an extremely fast result.
Otherwise the update is correct, data is consistent across JSON and markdown, and the automation is working as designed. No issues.
1 participant
Automated Benchmark Update
This PR updates the benchmark documentation with the latest results from the Speed Comparison workflow.
Benchmarks Produced
Individual benchmark artifacts are available for download:
benchmark-DataDrivenTestsbenchmark-AsyncTestsbenchmark-ScaleTestsbenchmark-MatrixTestsbenchmark-MassiveParallelTestsbenchmark-SetupTeardownTestsbenchmark-BuildTimebenchmark-summary(aggregated overview)Runtime Benchmarks
Build Benchmarks
Changes
docs/static/benchmarks/latest.jsondocs/static/benchmarks/historical.jsondocs/docs/benchmarks/index.mddocs/static/benchmarks/summary.jsonWorkflow Run
🤖 This PR was automatically created and will be merged automatically once CI checks pass.