fix(benchmarks): make SetupTeardownTests JSON tests Native AOT compatible by thomhurst · Pull Request #6434 · thomhurst/TUnit

thomhurst · 2026-07-16T14:03:03Z

Problem

The README benchmark table shows — for TUnit (AOT) on the Setup/teardown lifecycle row. The AOT benchmark run for SetupTeardownTests crashes:

JsonOperationTest / AsyncJsonOperationTest serialize an anonymous type with System.Text.Json.JsonSerializer.Serialize(data).
PublishAot=true sets JsonSerializerIsReflectionEnabledByDefault=false, so reflection-based STJ throws InvalidOperationException at runtime.
2 failing tests → non-zero exit → CliWrap ExecuteBufferedAsync throws in RuntimeBenchmarks.TUnit_AOT → BenchmarkDotNet records NA → README renders —.

This is a benchmark-suite bug, not a TUnit hooks bug — [Before(Test)]/[After(Test)] work fine under AOT.

Fix

Replace the anonymous type with a concrete JsonPayload record serialized via a source-generated JsonSerializerContext. Works under Native AOT and compiles for all four framework variants (shared file).

Verification

No MSVC linker on this machine, so full AOT link wasn''t possible locally. Instead, reproduced the exact AOT failure mode by building the TUnit variant with -p:JsonSerializerIsReflectionEnabledByDefault=false (the same feature switch PublishAot flips) and running --treenode-filter "/*/*/SetupTeardownTests/*":

Confirmed "System.Text.Json.JsonSerializer.IsReflectionEnabledByDefault": false in runtimeconfig.json
22/22 tests pass

Also compile-checked XUNIT3, NUNIT, MSTEST variants — all build clean.

After merge, the next weekly Speed Comparison run should fill in the missing AOT cell.

…ible JsonOperationTest and AsyncJsonOperationTest serialized an anonymous type via reflection-based System.Text.Json, which is disabled under PublishAot (JsonSerializerIsReflectionEnabledByDefault=false). Both tests threw at runtime, failing the run, so BenchmarkDotNet reported NA for TUnit AOT in the Setup/teardown lifecycle scenario and the README showed a dash. Replace the anonymous type with a concrete record and a source-generated JsonSerializerContext, which works under Native AOT and all four frameworks. Verified by running the suite with the reflection feature switch disabled: 22/22 pass.

codacy-production · 2026-07-16T14:05:23Z

Not up to standards ⛔

🔴 Issues 1 minor

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category Results

CodeStyle 1 minor

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer}
_{TIP This summary will be updated as you push new changes.}

claude · 2026-07-16T14:08:22Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Summary: This PR replaces an anonymous-type JSON payload in tools/speed-comparison/UnifiedTests/SetupTeardownTests.cs with a concrete JsonPayload record serialized via a source-generated JsonSerializerContext, fixing a Native AOT crash (reflection-based System.Text.Json is disabled under PublishAot). Verified the fix is correctly scoped (benchmark tooling, not core engine — CLAUDE.md's Dual-Mode/Snapshot Testing rules don't apply here), directly satisfies the AOT Compatible rule, compiles cleanly, and preserves the original serialization semantics (same field names/values, no behavioral regression).

Updated [TUnit](https://github.com/thomhurst/TUnit) from 1.59.0 to 1.61.15. <details> <summary>Release notes</summary> _Sourced from [TUnit's releases](https://github.com/thomhurst/TUnit/releases)._ ## 1.61.15  ## What's Changed ### Other Changes * feat: Allow disabling the automatic html report upload to upload-artifacts@v7 by @Crashdummyy in thomhurst/TUnit#6447 * fix: stop retrying artifact upload on non-retryable status codes by @thomhurst in thomhurst/TUnit#6449 ### Dependencies * chore(deps): update tunit to 1.61.0 by @thomhurst in thomhurst/TUnit#6437 * chore(deps): update verify to 31.24.3 by @thomhurst in thomhurst/TUnit#6438 * chore(deps): update dependency opentelemetry.instrumentation.aspnetcore to 1.17.0 by @thomhurst in thomhurst/TUnit#6440 * chore(deps): update verify to 31.25.0 by @thomhurst in thomhurst/TUnit#6442 * chore(deps): update dependency docusaurus-plugin-llms to ^0.5.0 by @thomhurst in thomhurst/TUnit#6443 * chore(deps): update opentelemetry to 1.17.0 by @thomhurst in thomhurst/TUnit#6441 ## New Contributors * @Crashdummyy made their first contribution in thomhurst/TUnit#6447 **Full Changelog**: thomhurst/TUnit@v1.61.0...v1.61.15 ## 1.61.0  ## What's Changed ### Other Changes * docs: revamp README with feature showcase and auto-updating benchmarks by @thomhurst in thomhurst/TUnit#6430 * docs: align docs homepage messaging with revamped README by @thomhurst in thomhurst/TUnit#6431 * fix(engine): emit ECMA-335 metadata-format type names in TestMethodIdentifierProperty by @thomhurst in thomhurst/TUnit#6433 * fix(benchmarks): make SetupTeardownTests JSON tests Native AOT compatible by @thomhurst in thomhurst/TUnit#6434 * Organize repository into conventional .NET layout by @thomhurst in thomhurst/TUnit#6435 ### Dependencies * chore(deps): update dependency polyfill to v11 by @thomhurst in thomhurst/TUnit#6418 * chore(deps): update dependency tunit.aspire to 1.6* by @thomhurst in thomhurst/TUnit#6419 * chore(deps): update tunit to 1.6* by @thomhurst in thomhurst/TUnit#6420 * chore(deps): update dependency polyfill to 11.0.1 by @thomhurst in thomhurst/TUnit#6421 * chore(deps): update dependency polyfill to 11.0.1 by @thomhurst in thomhurst/TUnit#6422 * chore(deps): update verify to 31.24.2 by @thomhurst in thomhurst/TUnit#6424 * chore(deps): update microsoft.build to 18.8.2 by @thomhurst in thomhurst/TUnit#6426 * chore(deps): update dependency microsoft.net.stringtools to 18.8.2 by @thomhurst in thomhurst/TUnit#6425 * chore(deps): update actions/setup-dotnet action to v6 by @thomhurst in thomhurst/TUnit#6427 * chore(deps): bump websocket-driver from 0.7.4 to 0.7.5 in /docs by @dependabot[bot] in thomhurst/TUnit#6429 * chore(deps): update opentelemetry to 1.17.0 by @thomhurst in thomhurst/TUnit#6432 **Full Changelog**: thomhurst/TUnit@v1.60.0...v1.61.0 ## 1.60.0  ## What's Changed ### Other Changes * feat: cross-process test report aggregation — merged HTML report + single GitHub summary by @thomhurst in thomhurst/TUnit#6415 ### Dependencies * chore(deps): update tunit to 1.59.0 by @thomhurst in thomhurst/TUnit#6398 * chore(deps): update verify to 31.24.1 by @thomhurst in thomhurst/TUnit#6399 * chore(deps): update microsoft.testing to 2.3.2 by @thomhurst in thomhurst/TUnit#6401 * chore(deps): update mstest to 4.3.2 by @thomhurst in thomhurst/TUnit#6402 * chore(deps): update actions/setup-node action to v7 by @thomhurst in thomhurst/TUnit#6403 * chore(deps): update dependency microsoft.net.test.sdk to 18.8.0 by @thomhurst in thomhurst/TUnit#6404 * chore(deps): update dependency microsoft.net.test.sdk to 18.8.1 by @thomhurst in thomhurst/TUnit#6405 * chore(deps): update dependency fsharp.core to 10.1.302 by @thomhurst in thomhurst/TUnit#6406 * chore(deps): update dependency microsoft.templateengine.authoring.cli to v10.0.302 by @thomhurst in thomhurst/TUnit#6407 * chore(deps): update dependency microsoft.entityframeworkcore to 10.0.10 by @thomhurst in thomhurst/TUnit#6410 * chore(deps): update dependency dotnet-sdk to v10.0.302 by @thomhurst in thomhurst/TUnit#6409 * chore(deps): update dependency system.commandline to 2.0.10 by @thomhurst in thomhurst/TUnit#6412 * chore(deps): update dependency microsoft.templateengine.authoring.templateverifier to 10.0.302 by @thomhurst in thomhurst/TUnit#6411 * chore(deps): update microsoft.aspnetcore to 10.0.10 by @thomhurst in thomhurst/TUnit#6413 * chore(deps): update microsoft.extensions to 10.0.10 by @thomhurst in thomhurst/TUnit#6414 * chore(deps): update microsoft.extensions to 10.8.0 by @thomhurst in thomhurst/TUnit#6416 * chore(deps): update dependency polyfill to v11 by @thomhurst in thomhurst/TUnit#6417 **Full Changelog**: thomhurst/TUnit@v1.59.0...v1.60.0 Commits viewable in [compare view](thomhurst/TUnit@v1.59.0...v1.61.15). </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=TUnit&package-manager=nuget&previous-version=1.59.0&new-version=1.61.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Updated [TUnit.Core](https://github.com/thomhurst/TUnit) from 1.59.0 to 1.61.15. <details> <summary>Release notes</summary> _Sourced from [TUnit.Core's releases](https://github.com/thomhurst/TUnit/releases)._ ## 1.61.15  ## What's Changed ### Other Changes * feat: Allow disabling the automatic html report upload to upload-artifacts@v7 by @Crashdummyy in thomhurst/TUnit#6447 * fix: stop retrying artifact upload on non-retryable status codes by @thomhurst in thomhurst/TUnit#6449 ### Dependencies * chore(deps): update tunit to 1.61.0 by @thomhurst in thomhurst/TUnit#6437 * chore(deps): update verify to 31.24.3 by @thomhurst in thomhurst/TUnit#6438 * chore(deps): update dependency opentelemetry.instrumentation.aspnetcore to 1.17.0 by @thomhurst in thomhurst/TUnit#6440 * chore(deps): update verify to 31.25.0 by @thomhurst in thomhurst/TUnit#6442 * chore(deps): update dependency docusaurus-plugin-llms to ^0.5.0 by @thomhurst in thomhurst/TUnit#6443 * chore(deps): update opentelemetry to 1.17.0 by @thomhurst in thomhurst/TUnit#6441 ## New Contributors * @Crashdummyy made their first contribution in thomhurst/TUnit#6447 **Full Changelog**: thomhurst/TUnit@v1.61.0...v1.61.15 ## 1.61.0  ## What's Changed ### Other Changes * docs: revamp README with feature showcase and auto-updating benchmarks by @thomhurst in thomhurst/TUnit#6430 * docs: align docs homepage messaging with revamped README by @thomhurst in thomhurst/TUnit#6431 * fix(engine): emit ECMA-335 metadata-format type names in TestMethodIdentifierProperty by @thomhurst in thomhurst/TUnit#6433 * fix(benchmarks): make SetupTeardownTests JSON tests Native AOT compatible by @thomhurst in thomhurst/TUnit#6434 * Organize repository into conventional .NET layout by @thomhurst in thomhurst/TUnit#6435 ### Dependencies * chore(deps): update dependency polyfill to v11 by @thomhurst in thomhurst/TUnit#6418 * chore(deps): update dependency tunit.aspire to 1.6* by @thomhurst in thomhurst/TUnit#6419 * chore(deps): update tunit to 1.6* by @thomhurst in thomhurst/TUnit#6420 * chore(deps): update dependency polyfill to 11.0.1 by @thomhurst in thomhurst/TUnit#6421 * chore(deps): update dependency polyfill to 11.0.1 by @thomhurst in thomhurst/TUnit#6422 * chore(deps): update verify to 31.24.2 by @thomhurst in thomhurst/TUnit#6424 * chore(deps): update microsoft.build to 18.8.2 by @thomhurst in thomhurst/TUnit#6426 * chore(deps): update dependency microsoft.net.stringtools to 18.8.2 by @thomhurst in thomhurst/TUnit#6425 * chore(deps): update actions/setup-dotnet action to v6 by @thomhurst in thomhurst/TUnit#6427 * chore(deps): bump websocket-driver from 0.7.4 to 0.7.5 in /docs by @dependabot[bot] in thomhurst/TUnit#6429 * chore(deps): update opentelemetry to 1.17.0 by @thomhurst in thomhurst/TUnit#6432 **Full Changelog**: thomhurst/TUnit@v1.60.0...v1.61.0 ## 1.60.0  ## What's Changed ### Other Changes * feat: cross-process test report aggregation — merged HTML report + single GitHub summary by @thomhurst in thomhurst/TUnit#6415 ### Dependencies * chore(deps): update tunit to 1.59.0 by @thomhurst in thomhurst/TUnit#6398 * chore(deps): update verify to 31.24.1 by @thomhurst in thomhurst/TUnit#6399 * chore(deps): update microsoft.testing to 2.3.2 by @thomhurst in thomhurst/TUnit#6401 * chore(deps): update mstest to 4.3.2 by @thomhurst in thomhurst/TUnit#6402 * chore(deps): update actions/setup-node action to v7 by @thomhurst in thomhurst/TUnit#6403 * chore(deps): update dependency microsoft.net.test.sdk to 18.8.0 by @thomhurst in thomhurst/TUnit#6404 * chore(deps): update dependency microsoft.net.test.sdk to 18.8.1 by @thomhurst in thomhurst/TUnit#6405 * chore(deps): update dependency fsharp.core to 10.1.302 by @thomhurst in thomhurst/TUnit#6406 * chore(deps): update dependency microsoft.templateengine.authoring.cli to v10.0.302 by @thomhurst in thomhurst/TUnit#6407 * chore(deps): update dependency microsoft.entityframeworkcore to 10.0.10 by @thomhurst in thomhurst/TUnit#6410 * chore(deps): update dependency dotnet-sdk to v10.0.302 by @thomhurst in thomhurst/TUnit#6409 * chore(deps): update dependency system.commandline to 2.0.10 by @thomhurst in thomhurst/TUnit#6412 * chore(deps): update dependency microsoft.templateengine.authoring.templateverifier to 10.0.302 by @thomhurst in thomhurst/TUnit#6411 * chore(deps): update microsoft.aspnetcore to 10.0.10 by @thomhurst in thomhurst/TUnit#6413 * chore(deps): update microsoft.extensions to 10.0.10 by @thomhurst in thomhurst/TUnit#6414 * chore(deps): update microsoft.extensions to 10.8.0 by @thomhurst in thomhurst/TUnit#6416 * chore(deps): update dependency polyfill to v11 by @thomhurst in thomhurst/TUnit#6417 **Full Changelog**: thomhurst/TUnit@v1.59.0...v1.60.0 Commits viewable in [compare view](thomhurst/TUnit@v1.59.0...v1.61.15). </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=TUnit.Core&package-manager=nuget&previous-version=1.59.0&new-version=1.61.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

thomhurst temporarily deployed to Pull Requests July 16, 2026 14:03 — with GitHub Actions Inactive

thomhurst enabled auto-merge (squash) July 16, 2026 14:06

thomhurst disabled auto-merge July 16, 2026 14:08

thomhurst merged commit 21301a6 into main Jul 16, 2026
12 of 13 checks passed

thomhurst deleted the fix/benchmark-setupteardown-aot-json branch July 16, 2026 14:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(benchmarks): make SetupTeardownTests JSON tests Native AOT compatible#6434

fix(benchmarks): make SetupTeardownTests JSON tests Native AOT compatible#6434
thomhurst merged 1 commit into
mainfrom
fix/benchmark-setupteardown-aot-json

thomhurst commented Jul 16, 2026

Uh oh!

codacy-production Bot commented Jul 16, 2026

Uh oh!

claude Bot commented Jul 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

thomhurst commented Jul 16, 2026

Problem

Fix

Verification

Uh oh!

codacy-production Bot commented Jul 16, 2026

Not up to standards ⛔

Uh oh!

claude Bot commented Jul 16, 2026

Code review

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant