Skip to content

🤖 Update Mock Benchmark Results#6315

Merged
thomhurst merged 1 commit into
mainfrom
automated-mock-benchmarks-update
Jun 25, 2026
Merged

🤖 Update Mock Benchmark Results#6315
thomhurst merged 1 commit into
mainfrom
automated-mock-benchmarks-update

Conversation

@thomhurst

Copy link
Copy Markdown
Owner

Automated Mock Benchmark Update

This PR updates the mock benchmark documentation with the latest results from the Mock Benchmarks workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

  • mock-benchmark-MockCreation
  • mock-benchmark-Setup
  • mock-benchmark-Invocation
  • mock-benchmark-Verification
  • mock-benchmark-Callback
  • mock-benchmark-CombinedWorkflow
  • mock-benchmark-summary (aggregated overview)

Libraries Compared

Libraries are discovered dynamically from the benchmark output. See the generated summary.json for the full list.

Changes

  • Updated mock benchmark data in docs/static/benchmarks/mocks/latest.json
  • Regenerated mock benchmark documentation in docs/docs/benchmarks/mocks/
  • Updated mock benchmark summary in docs/static/benchmarks/mocks/summary.json

Workflow Run

  • Run ID: 28144357695
  • Triggered: schedule

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

@thomhurst thomhurst added automated benchmarks documentation Improvements or additions to documentation ignore-for-release labels Jun 25, 2026
@thomhurst
thomhurst merged commit 2cd32f2 into main Jun 25, 2026
11 of 13 checks passed
@thomhurst
thomhurst deleted the automated-mock-benchmarks-update branch June 25, 2026 03:27
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@claude

claude Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Code Review

This is an automated benchmark documentation update (no production code changes). The review covers the data quality and patterns in the update.

Summary

All changes are documentation-only: updated benchmark numbers, dates, and Mermaid chart data across the mock benchmark markdown files and JSON data files.

Observations

Data consistency: The Mermaid upper bounds are recalculated correctly relative to the new data (e.g., Callback drops from 222,976 ns to 169,061 ns as Moq's result improved).

Performance trends in this run:

  • TUnit.Mocks shows a slight regression in some categories (e.g., Callback: 659 ns → 807 ns, ~22%), but the same trend appears across all source-generated/proxy libraries in this category, which is consistent with expected daily CI benchmark variance rather than a true regression.
  • Moq's Callback result improved significantly (185,813 ns → 140,883 ns), which also suggests normal run-to-run variation in the CI environment rather than a code change.

No issues found with the automated update itself. The data appears consistently applied across all files (Markdown + JSON).

Suggestion (non-blocking, for future automation improvement)

Since these benchmarks run daily on a shared CI runner, noticeable run-to-run variance (~20%) can make it hard to spot real regressions. Consider tracking a rolling average (e.g., 7-day mean) in alongside the latest results. This would make genuine performance regressions easier to distinguish from noise without adding overhead to individual benchmark runs.

Overall, this automated PR is working correctly. ✓

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated benchmarks documentation Improvements or additions to documentation ignore-for-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant