Measure Inspect Web runtime performance and migrate CoreCLR to .NET 12 R2R

The project mission is to build robust, capable features that provide foundational capabilities or compelling user experiences, with results that are conventionally sound, delightfully new or unique, or both. This work uses production-site evidence to choose the Inspect Web runtime configuration rather than treating runtime availability as sufficient justification.

Goal

Establish a repeatable deployed-site benchmark and scheduled evidence lane, retain the Mono .NET 11 site as the control, migrate the isolated CoreCLR site to a pinned runtime-main .NET 12 cohort without ReadyToRun, and then measure the same site with application ReadyToRun enabled.

This issue is the end-to-end tracker. The deployed Inspect Web sites are the production consumers; the benchmark harness and nightly workflow are test-infrastructure hosts. The user explicitly approved browser-only scope for this runtime comparison.

Method

The harness records the site URL, product build identity, browser and host identity, exact package coordinate, scenario parameters, individual samples, and summary statistics. It validates semantic outcomes as well as timings.

Measurements are separated by what they include:

cold site startup: navigation through managed runtime readiness, including deployed asset transfer;
cold package inspection: real user-facing acquisition plus managed projection, reported as network-sensitive evidence;
warm package projection: same package after process-local acquisition;
whole-package performance analysis: first-use latency and repeated warm latency;
member-facts batch: sustained throughput over a deterministic set of distinct methods;
method-body comparison: first-use and repeated warm latency over a fixed pair with validated C# and IL results.

Each sample uses a fresh browser context. Site order alternates between rounds. Cross-site comparison requires matching product commit identities by default; an explicit diagnostic override may collect non-comparable data without presenting it as a baseline.

Pinned initial coordinate: [email protected], net10.0. It exercises package acquisition, metadata projection, IL analysis, decompilation, and diff production. The current unsupported .NET 11 CoreCLR deployment can exceed its 30-second browser package-operation deadline even for this coordinate; those failures are product evidence and make the report non-comparable.

Adoption plan: five slices

Complete — Add deployed Inspect Web runtime benchmark #6091. Add the deployed-site performance harness, document its contract, and prove accepted and rejected outcomes against the existing Mono and CoreCLR deployments.
Complete — Add nightly Inspect Web runtime performance evidence #6160. Add a nightly performance workflow that runs both sites interleaved on the same runner, retains raw JSON artifacts, records host load, and emits no trend point when build identities, semantic results, or product operations fail.
Move only the isolated CoreCLR deployment to one exact runtime-main .NET 12 SDK/workload cohort with ReadyToRun disabled; retain Mono .NET 11 unchanged and run the same benchmark.
Enable non-composite application ReadyToRun on the CoreCLR deployment after selecting a daily containing the Linux R2R path fix; preserve runtime-async and deployment correctness gates.
Compare all three recorded configurations, including compressed framework size, startup latency, expensive-operation latency, sustained member-analysis throughput, failures/timeouts, and semantic-result equality; decide whether R2R remains enabled.

Current evidence

A matched-head one-sample corrected run completed every stage on both sites with equal semantic fingerprints. A corrected three-sample run was rejected after CoreCLR reached its product package-operation deadline in all three samples. Earlier successful samples showed materially higher CoreCLR latency for package projection, analysis throughput, and method-body comparison.

These results prove the harness contract but are not a statistically useful runtime baseline. The nightly lane owns repeated collection on a controlled host; rejected runs remain visible rather than being retried into success or reduced to partial timing comparisons.

Non-goals

No performance threshold is chosen before stable matched-head nightly evidence exists.
No claim that network-sensitive package acquisition isolates runtime CPU performance.
No replacement of the Mono deployment during this investigation.
No floating daily SDK or mixed SDK/runtime/workload cohort.
No increase to product operation deadlines merely to make the benchmark complete.

Slice 2 complete — #6160 merged

The nightly evidence lane is now on main at merge commit be85c75f2c7ac5b425c2af2990d48ece75ca7282. It runs Mono and CoreCLR interleaved on one runner, records host load, retains raw reports and logs for 90 days, and emits hash-linked trend points only for fully comparable runs.

Next within the Inspect Web runtime performance evidence theme: slice 3 moves only the isolated CoreCLR deployment to one pinned runtime-main .NET 12 SDK/workload cohort with ReadyToRun disabled, while Mono .NET 11 remains the control.

Slice 3 is now in #6218 at head f6c01425ba695536ba1166b66b1c6aa6cb96a261: the isolated CoreCLR deployment is pinned to the coherent .NET 12 runtime-main cohort without ReadyToRun. The workflow records SDK/workload evidence and verifies that the published native runtime bytes match the pinned CoreCLR browser pack; Mono remains the .NET 11 control.

Accepted post-recovery production comparison: run 34650478741.

Both Mono and CoreCLR served product commit 68fa91e75cea4f089b9ac754e0d109c9b7783ab4; each completed 5/5 samples with zero failures. comparison.comparable, buildComparable, and semanticResultsEquivalent are all true, with no rejection reasons. The trend receipt's sourceReportSha256 matches the retained raw report: deb775b6997c67c3e99decb1f33eb0667fb198611d5aff21901267ac21edde6a.

Metric	Mono median	CoreCLR median	CoreCLR / Mono
Startup	1,532 ms	2,164 ms	1.41x
Framework transfer	5,621,146 B	6,057,535 B	1.08x
Cold package	3,441 ms	9,007 ms	2.62x
Warm package	744 ms	4,980 ms	6.69x
First package-performance query	3,287 ms	12,107 ms	3.68x
Warm package-performance query	2,099 ms	11,041 ms	5.26x
Member throughput	1.869 ops/s	0.206 ops/s	0.11x
First method comparison	3,629 ms	10,153 ms	2.80x
Warm method comparison	1,720 ms	8,361 ms	4.86x

Normalized one-minute host load was 0.560 per logical processor before the run and 0.395 after it. This is an accepted matched production result, not an R2R trend: the R2R configuration was rejected for correctness before performance measurement. Retained local copy: session artifact runtime-comparison-34650478741.

The runtime-performance plan is now organized around one controlled nightly cohort rather than permanent public staging sites.

The first slice builds Mono .NET 11, CoreCLR .NET 12 IL, and CoreCLR .NET 12 R2R from one source commit and shared frontend artifact. Deterministic admission requires Mono and CoreCLR IL; R2R is timed only when its real Worker/package operation passes. An explicitly marked R2R product-correctness rejection is retained as evidence, while infrastructure or evaluation failures make the run red. All admitted variants are then benchmarked sequentially over loopback on one runner.

The existing deployed-site nightly remains useful as a secondary production synthetic for deployment, CDN, TLS, and browser-path effects, but it is no longer the primary runtime comparison.

Remaining adoption path:

Land the controlled cohort lane and typed, hash-bound receipts.
Add policy-driven advancement of the checked-in .NET 12 daily SDK/runtime/workload/VMR pin.
Aggregate accepted and rejected cohort receipts into durable history before the 90-day workflow artifacts expire, then define the decision policy for promoting R2R when correctness and performance evidence support it.

Fixed-head local evidence for the first slice admitted Mono and CoreCLR IL with equivalent semantics and classified CoreCLR R2R as a correctness rejection through INSPECT_WEB_PRODUCT_OPERATION_FAILURE:producer-contract.

Slice 2 is now in #6888 at exact head 8debebda5d47d726b2b2474a6751942f750a7ac7.

Controlled cohort run https://github.com/richlander/dotnet-inspect/actions/runs/34799638871 accepted the new daily identity: SDK 12.0.100-alpha.1.26462.105, runtime/workload 12.0.0-alpha.1.26462.105, VMR c8c7ac15e5a07158cc2d5199d25d1840eaadf18e. Mono and CoreCLR IL passed the real package-operation admission and five-sample comparison. CoreCLR R2R still reached the product operation and failed with INSPECT_WEB_PRODUCT_OPERATION_FAILURE:producer-contract / index out of bounds, so the accepted cohort remains two-way.

#6888 adds the shared checked-in pin, strict authoritative discovery and workload/VMR validation, reusable candidate-cohort admission, and the daily pin-only proposal branch plus tracker boundary.

Post-#7109 runtime-pin confirmation on main reached the candidate cohort and exposed a new candidate-specific blocker.

Run: https://github.com/richlander/dotnet-inspect/actions/runs/35028262151

Validated candidate identity:

SDK: 12.0.100-alpha.1.26465.103
runtime/workload: 12.0.0-alpha.1.26465.103
dotnet/dotnet VMR: 62f3d29f24b39b1368b02b9449e8b974f8e6c132

The SDK-isolation repair is confirmed: discovery, exact SDK installation and selection, wasm-tools installation, workload/pack/VMR identity validation, Mono publication, and CoreCLR IL publication all succeeded. CoreCLR ReadyToRun then failed deterministically during publication:

System.OverflowException: Arithmetic operation resulted in an overflow.
  at ILCompiler.DependencyAnalysis.ReadyToRun.InliningInfoNode.GetData(...)
  InliningInfoNode.cs:line 203
NETSDK1096: Optimizing assemblies for performance failed.

The failing source line enumerates the sorted inliners while serializing InliningInfo2. Searches found no existing dotnet/runtime issue matching this exact InliningInfoNode.GetData overflow.

Classification: blocker requiring repair / upstream candidate regression, not the accepted browser-level ReadyToRun rejection. The workflow correctly stopped before admission, measurement, or proposal publication; no automation/inspect-web-runtime-pin-* branch was created. The pinned 12.0.100-alpha.1.26462.105 cohort remains healthy, including successful runtime-site run https://github.com/richlander/dotnet-inspect/actions/runs/35028259025 on the same product commit.

No rerun was requested because the compiler overflow is deterministic and the owning contract requires an evaluated R2R result rather than treating a build failure as an accepted rejection.

Runtime-pin candidate 12.0.100-alpha.1.26512.139 is rejected; the checked-in 12.0.100-alpha.1.26462.105 cohort remains authoritative.

Scheduled discovery correctly stopped when wasm-tools changed from the reviewed net11 logical pack inventory to net12:
https://github.com/richlander/dotnet-inspect/actions/runs/38399047563
A fixed-head policy candidate was tested through the controlled cohort at cd9e1694c3d9046fcf4e78a3dfeb6b3925db8636:
https://github.com/richlander/dotnet-inspect/actions/runs/38444384088
Mono built successfully. Both CoreCLR variants failed before admission because the net11.0 project requires wasm-tools-net11, not the candidate wasm-tools workload.
The candidate wasm-tools-net11 manifest is not a coherent substitute: its five direct packs are version 11.0.0-rc.1.26453.118, not candidate runtime 12.0.0-alpha.1.26512.139.

Classification: expected policy rejection caused by an upstream candidate incompatibility. Advancing to the net12 logical pack family would abandon the owned net11.0 target; selecting wasm-tools-net11 would abandon the exact .NET 12 runtime cohort. No rerun or pin update is appropriate, and no proposal branch was created.

Comment authored with GitHub Copilot CLI.

Nightly operations, 2026-11-15 UTC: runtime-pin discovery is blocked before cohort admission, not a successful no-newer-candidate result. Scheduled run 43593279667 attempt 1 at 3dbe19edc9326ca89638cb163a884f6180bc29f9 fails with Error: Candidate wasm-tools workload packs changed. in runtime-cohort-pin-model.ts:521. No proposal branch exists. First proof: https://github.com/richlander/dotnet-inspect/actions/runs/43593279667/job/132442347530 . Classify as blocker requiring runtime-acquisition contract review/repair, not the explicitly admitted CoreCLR R2R thunk rejection. No unchanged rerun or pin/branch mutation performed.

Comparison deployment 43582137855 attempt 1 rejects candidate 43565775734 attempt 1 because its Windows certification is completed/cancelled. This is correct candidate-boundary rejection; repair requires complete replacement candidate evidence, not bypassing validation. Proof: https://github.com/richlander/dotnet-inspect/actions/runs/43582137855/job/132408926279 .

The public synthetic performance workflow now has only workflow_dispatch on main; its absent daily schedule is intentional, not a missed nightly run. Controlled comparison evidence could not run because candidate selection failed.

Authored by GitHub Copilot, powered by GPT-6.1 Sol (gpt-6.1-sol).

Navigation Menu

Repository navigation

Measure Inspect Web runtime performance and migrate CoreCLR to .NET 12 R2R #6077

Goal

Method

Adoption plan: five slices

Current evidence

Non-goals

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Measure Inspect Web runtime performance and migrate CoreCLR to .NET 12 R2R #6077

Description

Goal

Method

Adoption plan: five slices

Current evidence

Non-goals

Activity

richlander commented on Sep 6, 2026

richlander commented on Sep 6, 2026

richlander commented on Sep 11, 2026

richlander commented on Sep 13, 2026

richlander commented on Sep 14, 2026

richlander commented on Sep 15, 2026

richlander commented on Oct 13, 2026

richlander commented on Nov 15, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions