Improve LoggerMessageGenerator and JsonSourceGenerator incrementality#124077
Improve LoggerMessageGenerator and JsonSourceGenerator incrementality#124077
Conversation
Removed Compilation from generator pipelines to improve incrementality. - LoggerMessageGenerator: Extract well-known symbols in ForAttributeWithMetadataName transform - JsonSourceGenerator: Move KnownTypeSymbols creation into ForAttributeWithMetadataName transform - Both generators now avoid Combine with CompilationProvider Co-authored-by: cincuranet <[email protected]>
- Remove conditional using directive for GetBestTypeByMetadataName extension - Update Roslyn 3.11 generator to use new Parser and Emitter APIs - Both generators now build successfully Co-authored-by: cincuranet <[email protected]>
- Change IsBaseOrIdentity to take Compilation instead of SemanticModel - Simplify Execute method by removing unnecessary grouping - Add clarifying comment in Roslyn 3.11 generator Co-authored-by: cincuranet <[email protected]>
src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/LoggerMessageGenerator.Roslyn4.0.cs
Outdated
Show resolved
Hide resolved
|
@copilot is it possible to get the benchmark numbers showing the improvement? |
Move all parsing logic into ForAttributeWithMetadataName transform to extract value-based LoggerClass data. No longer pass SemanticModel or INamedTypeSymbol references through the incremental pipeline, which was causing cache misses. Parser now runs in the transform and returns LoggerClass instances directly. Execute step merges classes and emits code without needing symbols. Co-authored-by: tarekgh <[email protected]>
The improvement is in build-time incrementality rather than runtime performance. The generator now avoids full re-runs when unrelated code changes (e.g., changing a method body that doesn't affect logger attributes). To measure this, you'd need to:
Roslyn's incremental generator infrastructure doesn't expose built-in metrics for this. The improvement is structural - by removing Would you like me to create a test project demonstrating the reduced re-runs, or would the structural fix verification be sufficient? |
src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/LoggerMessageGenerator.Parser.cs
Show resolved
Hide resolved
src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/LoggerMessageGenerator.Roslyn4.0.cs
Outdated
Show resolved
Hide resolved
…ural equality - Add LoggerClassSpec, LoggerMethodSpec, LoggerParameterSpec as immutable records - Implement IEquatable with structural equality using ImmutableEquatableArray - Keep original mutable classes for parsing, convert to Spec in transform - Add FromSpec method to convert back to mutable classes for emission - Include required polyfills (IsExternalInit, RequiredMember, HashHelpers, etc.) Co-authored-by: tarekgh <[email protected]>
- Use Key.Equals and Value.Equals instead of EqualityComparer - Remove unnecessary null-conditional operators in GetHashCode - Consistent with IEquatable constraints Co-authored-by: tarekgh <[email protected]>
|
@copilot please fix the failure: Discovering: Microsoft.Extensions.Logging.Generators.Roslyn3.11.Tests (method display = ClassAndMethod, method display options = None) |
The Roslyn 3.11 generator was silently returning when System.Exception was unavailable, instead of reporting the MissingRequiredType diagnostic. Split the combined null check to handle exceptionSymbol separately, matching the original Parser behavior that reported the diagnostic before returning. Co-authored-by: tarekgh <[email protected]>
Fixed in bfe3713. The Roslyn 3.11 path had a combined null check that silently returned when |
|
@copilot run the code review skill |
Code review completed. Here are the findings: No ❌ errors found.
Line 102 in context.RegisterSourceOutput(loggerClasses.Collect(), static (spc, items) => Execute(items, spc));Using
Lines 151-155: When merging partial class methods, a new 💡 suggestion — Lines 182-194: 💡 suggestion — The spec types are declared as ✅ Verified correct:
Code review completed. Found and fixed one issue in fdae4aa:
💡 TemplateMap ordering (follow-up, not blocking) — No errors found. All other aspects look correct:
|
All items from the same compilation should agree on hasStringCreate, but using |= instead of = is more resilient and costs nothing. Co-authored-by: tarekgh <[email protected]>
Restore the original pattern where KnownTypeSymbols is created via CompilationProvider to maintain 1-1 correspondence between Compilation and KnownTypeSymbols instances, avoiding performance regression. Co-authored-by: eiriktsarpalis <[email protected]>
|
@tarekgh do you have a roslyn tracking issue for the We'll likely also need to file some breaking change notice since we document the pragma approach in https://learn.microsoft.com/en-us/dotnet/fundamentals/syslib-diagnostics/syslib1025 /cc @ericstj |
|
I found #92509 and dotnet/roslyn#68291 which have more background on the issue |
|
Right, #92509 is the tracking issue for different generators. I'll file the breaking change doc. |
📋 Breaking Change Documentation RequiredCreate a breaking change issue with AI-generated content Generated by Breaking Change Documentation Tool - 2026-02-10 15:02:39 |
|
I filed the breaking change doc and linked it to this PR. @akoeplinger thanks for pointing at the failures and sorry for the inconvenience. Please let me know if there is anything else I can help with. |
…4186) ## Description Reverts System.Text.Json changes from PR #124077 per @eiriktsarpalis feedback. The change caused a performance regression by breaking the 1-1 correspondence between `Compilation` and `KnownTypeSymbols` instances, creating fresh instances for every `JsonSerializerContext` instead of caching per compilation. LoggerMessageGenerator changes remain unchanged. ## Changes Restored the original incremental pipeline pattern in `JsonSourceGenerator.Roslyn4.0.cs`: **Before (PR #124077):** ```csharp contextGenerationSpecs = context.SyntaxProvider .ForAttributeWithMetadataName( ..., (context, cancellationToken) => { var knownTypeSymbols = new KnownTypeSymbols(context.SemanticModel.Compilation); Parser parser = new(knownTypeSymbols); // ... parsing }) ``` **After (this PR):** ```csharp IncrementalValueProvider<KnownTypeSymbols> knownTypeSymbols = context.CompilationProvider .Select((compilation, _) => new KnownTypeSymbols(compilation)); contextGenerationSpecs = context.SyntaxProvider .ForAttributeWithMetadataName( ..., (context, _) => (ContextClass: context.TargetNode, context.SemanticModel)) .Combine(knownTypeSymbols) .Select(static (tuple, cancellationToken) => { Parser parser = new(tuple.Right); // KnownTypeSymbols // ... parsing using tuple.Left.ContextClass, tuple.Left.SemanticModel }) ``` This ensures `KnownTypeSymbols` is cached per compilation in the incremental pipeline, eliminating redundant type symbol resolution. ## Testing - Built clr+libs successfully - All 154 System.Text.Json.SourceGeneration unit tests pass <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > ## Context > > PR #124077 ("Improve LoggerMessageGenerator and JsonSourceGenerator incrementality") was merged and included changes to both the `LoggerMessageGenerator` and `JsonSourceGenerator`. However, per feedback from @eiriktsarpalis in #124077 (comment), the System.Text.Json changes should be reverted because: > > 1. Passing `Compilation` in the incremental graph does not automatically imply re-runs of the source generator every time it gets updated. This is only true in cases where such updates flow all the way to the `SourceProductionContext`. The STJ generator does not do this (source production is only triggered on the basis of equality of the resultant `ContextGenerationSpec`) and they have testing that validates that property. > 2. More importantly, the change actually **regresses performance**: `KnownTypeSymbols` defines a cache of known type symbols that the generator needs to reference. By removing it from the incremental graph, there is no longer a 1-1 correspondence between `Compilation` instances and `KnownTypeSymbols` instances, instead creating a fresh instance every time a `JsonSerializerContext` is processed for a given compilation. > > ## What needs to be done > > Revert ONLY the changes to `src/libraries/System.Text.Json/gen/JsonSourceGenerator.Roslyn4.0.cs` introduced by PR #124077 (merge commit `5709f35ed344f59062042d234b9a68683ec7e7a6`). > > The file was changed to: > - Remove the `CompilationProvider` dependency and the `knownTypeSymbols` incremental value provider > - Move `KnownTypeSymbols` instantiation into the `ForAttributeWithMetadataName` transform callback > - Restructure the pipeline to no longer use `.Combine(knownTypeSymbols)` and `.Select()` > > The revert should restore the original pattern where: > - `KnownTypeSymbols` is created via `context.CompilationProvider.Select(...)` as an `IncrementalValueProvider<KnownTypeSymbols>` > - The attribute provider uses `.Combine(knownTypeSymbols)` > - A subsequent `.Select()` performs the parsing using `tuple.Left` (for the context class + semantic model) and `tuple.Right` (for the known type symbols) > - The `ForAttributeWithMetadataName` transform only extracts `(ContextClass, SemanticModel)` without doing any parsing > > **Do NOT revert any LoggerMessageGenerator changes** — those are correct and should remain as-is. > </details> <!-- START COPILOT CODING AGENT SUFFIX --> *This pull request was created from Copilot chat.* > <!-- START COPILOT CODING AGENT TIPS --> --- ✨ Let Copilot coding agent [set things up for you](https://github.com/dotnet/runtime/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: eiriktsarpalis <[email protected]> Co-authored-by: Eirik Tsarpalis <[email protected]>
Description
Both generators were passing
Compilationthrough their pipelines, causing full re-runs on any compilation change rather than only when attributed syntax nodes changed. This PR refactors both generators to follow proper incremental generator patterns.Changes Made
LoggerMessageGenerator:
ForAttributeWithMetadataNametransform, returning immutable specs with structural equalityWithTrackingName(StepNames.LoggerMessageTransform)for Roslyn 4.4+MissingRequiredTypediagnostic whenSystem.Exceptionis unavailable in both Roslyn 3.11 and 4.0+ pathsJsonSourceGenerator:
CompilationProviderdependencyKnownTypeSymbolsinto transformTesting:
Pragma Suppression Behavioral Note
In the test
.targetsand.WithDiagnostics.csfiles, the PR adds project-level<NoWarn>forSYSLIB1002;SYSLIB1013;SYSLIB1018;8795and<WarningsNotAsErrors>8795</WarningsNotAsErrors>. This is becauseDiagnosticInfo.GetTrimmedLocationcreates locations withoutSyntaxTreereferences, breaking#pragma warning disablematching. While this only affects the test project directly, the sameDiagnosticInfopattern is used in the productionRoslyn 4.0 generatorpath, meaning end-users of theLoggerMessageGeneratorwill lose the ability to suppress diagnostics via#pragmadirectives. This is a behavioral regression. The existingSystem.Text.Jsongenerator already has this limitation too, so the Logging source generator will start to have the same behavior asSystem.Text.Json.Performance Impact
Generator caches pipeline values and only reprocesses when attributed methods change.
Testing
Original prompt
LoggerMessageGeneratorincrementality #76119✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.