Skip to content

Update compiler toolset to latest#69544

Closed
CyrusNajmabadi wants to merge 3 commits intodotnet:mainfrom
CyrusNajmabadi:updateToolset
Closed

Update compiler toolset to latest#69544
CyrusNajmabadi wants to merge 3 commits intodotnet:mainfrom
CyrusNajmabadi:updateToolset

Conversation

@CyrusNajmabadi
Copy link
Contributor

No description provided.

@CyrusNajmabadi CyrusNajmabadi requested a review from a team as a code owner August 16, 2023 16:53
@ghost ghost added Area-Infrastructure untriaged Issues and PRs which have not yet been triaged by a lead labels Aug 16, 2023
@RikkiGibson
Copy link
Member

RikkiGibson commented Aug 17, 2023

PRs between LKG #69568 and FKB #69569:
#68694 (most suspicious)
#69307
#69316
#69355

the bad codegen is occurring for an is-expression, so.

foreach (var fixAllContext in fixAllContexts)
{
Contract.ThrowIfFalse(fixAllContext.Scope is FixAllScope.Document or
FixAllScope.Project or FixAllScope.ContainingMember or FixAllScope.ContainingType);
await FixSingleContextAsync(fixAllContext, progressTracker, docIdToTextMerger).ConfigureAwait(false);
}

@jcouv
Copy link
Member

jcouv commented Aug 17, 2023

@RikkiGibson I reopened the two issues linked to the PR that was reverted. Could we provide any more details or a repro on the issue that made it necessary to revert? I see you linked to a source file. What was a the symptom (diagnostic, bad codegen, ...)?

@RikkiGibson
Copy link
Member

We have some screenshots from Teams which illustrate the unexpected difference in codegen for method DocumentBasedFixAllProviderHelpers.FixAllContextsAsync.

is-codegen-1
is-codegen-2

@RikkiGibson
Copy link
Member

also, duhh, just pushing the revert in source to here doesn't accomplish anything. We need to build with bootstrap compiler to observe any difference here.

@RikkiGibson
Copy link
Member

So far, I can manually bootstrap-build the Workspaces dll, and reproduce a failure in the unit test in question. Though, it looks like my stack trace is different locally:

[xUnit.net 00:00:07.32]     Microsoft.CodeAnalysis.UnitTests.BatchFixAllProviderTests.TestDefaultSelectionNestedFixers [FAIL]
  Failed Microsoft.CodeAnalysis.UnitTests.BatchFixAllProviderTests.TestDefaultSelectionNestedFixers [602 ms]
  Error Message:
   System.Collections.Generic.KeyNotFoundException : The given key 'Microsoft.NETFramework.ReferenceAssemblies.net472.1.0.2' was not present in the dictionary.
  Stack Trace:
     at System.Collections.Immutable.ImmutableDictionary`2.get_Item(TKey key)
   at Microsoft.CodeAnalysis.Testing.ReferenceAssemblies.<ResolveCoreAsync>d__50.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/ReferenceAssemblies.cs:line 333
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.CodeAnalysis.Testing.ReferenceAssemblies.<ResolveAsync>d__49.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/ReferenceAssemblies.cs:line 197
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.<CreateSolutionAsync>d__97.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs:line 1602
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.<CreateProjectImplAsync>d__95.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs:line 1417
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.<CreateProjectAsync>d__94.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs:line 1396
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.<GetSolutionAsync>d__93.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs:line 1374
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.<GetSortedDiagnosticsAsync>d__82.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs:line 1097
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.<VerifyDiagnosticsAsync>d__69.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs:line 465
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.Testing.CodeFixTest`1.<RunImplAsync>d__44.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.CodeFix.Testing/CodeFixTest`1.cs:line 305
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.<RunAsync>d__63.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs:line 188
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.UnitTests.BatchFixAllProviderTests.<TestDefaultSelectionNestedFixers>d__0.MoveNext() in C:\Users\rigibson\src\roslyn\src\Workspaces\CoreTest\BatchFixAllProviderTests.cs:line 46
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

in CI:

[xUnit.net 00:00:48.25]     Microsoft.CodeAnalysis.UnitTests.BatchFixAllProviderTests.TestDefaultSelectionNestedFixers [FAIL]
  Failed Microsoft.CodeAnalysis.UnitTests.BatchFixAllProviderTests.TestDefaultSelectionNestedFixers [17 s]
  Error Message:
   System.NullReferenceException : Object reference not set to an instance of an object.
  Stack Trace:
     at Microsoft.CodeAnalysis.CodeFixes.BatchFixAllProvider.<FixAllContextsAsync>d__4.MoveNext() in /_/src/Workspaces/Core/Portable/CodeFixes/FixAllOccurrences/BatchFixAllProvider.cs:line 57
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.CodeFixesAndRefactorings.DefaultFixAllProviderHelpers.<GetFixAsync>d__0`1.MoveNext() in /_/src/Workspaces/Core/Portable/CodeFixesAndRefactorings/DefaultFixAllProviderHelpers.cs:line 29
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.Testing.CodeFixTest`1.<FixAllAnalyerDiagnosticsInScopeAsync>d__54.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.CodeFix.Testing/CodeFixTest`1.cs:line 836
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.Testing.CodeFixTest`1.<VerifyFixAsync>d__48.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.CodeFix.Testing/CodeFixTest`1.cs:line 504
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.Testing.CodeFixTest`1.<VerifyFixAsync>d__46.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.CodeFix.Testing/CodeFixTest`1.cs:line 472
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.Testing.CodeFixTest`1.<RunImplAsync>d__44.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.CodeFix.Testing/CodeFixTest`1.cs:line 310
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.CodeAnalysis.Testing.AnalyzerTest`1.<RunAsync>d__63.MoveNext() in /_/src/Microsoft.CodeAnalysis.Testing/Microsoft.CodeAnalysis.Analyzer.Testing/AnalyzerTest`1.cs:line 188
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.UnitTests.BatchFixAllProviderTests.<TestDefaultSelectionNestedFixers>d__0.MoveNext() in /_/src/Workspaces/CoreTest/BatchFixAllProviderTests.cs:line 46
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

I don't yet grasp exactly what's going wrong and don't have a simplified repro.

@CyrusNajmabadi
Copy link
Contributor Author

Are you testing both in release mode?

@alrz
Copy link
Member

alrz commented Aug 18, 2023

I ran a debug build of Workspaces project with a debug build of compiler at the last commit of #68694 (be5c937)
(in the midst of updating the toolset I couldn't get a clean build off of main due to version mismatch on dependencies, but also to narrow the changes involved)

image

#68694 Looks like an innocent change by itself but it's probably a good idea to revert until #69194 can also pass bootstrap.

@RikkiGibson
Copy link
Member

Are you testing both in release mode?

Yes though I'm not convinced I have properly reproduced the issue locally.

@RikkiGibson
Copy link
Member

RikkiGibson commented Aug 18, 2023

I ended up downloading the Release mode test artifacts from this CI run and the same from a known good CI run, digging into ILSpy and getting the codegen for the MoveNext method where the issue occurs. I'm attaching the relevant artifacts and decompilation.

I found the diff was pretty substantial, too much to just screenshot or paste into a comment. The "good" codegen has IL code size 761 and the "bad" has code size 664. The source code for the FixAllContextsAsync method hasn't changed in 5 months, it looks like.

Also apologize that in some of these notes I seem to have mixed up which FixAllContextsAsync method is behaving badly. The stack trace from CI is the source of truth here :)

bad_codegen_repro.zip

@alrz
Copy link
Member

alrz commented Aug 18, 2023

I think the big difference in the bad codegen is the enumerator being captured in a local (instead of field)

This only happens for ImmutableArray<T> here:

   public async Task M(ImmutableArray<string> a) {
        foreach (var i in a) {
            M(i.Length is 10 or 20 or 30);
            await Task.FromResult(true).ConfigureAwait(false);
        }
    }

Weirdly enough, changing the array to List<T> correctly produces a field.

@RikkiGibson
Copy link
Member

RikkiGibson commented Aug 18, 2023

Thanks for the useful observation.

Hmm. I'm still not able to get a minimal reproducer. A program like the following doesn't NRE in either Debug or Release mode, using a compiler from Aug 17th.

using System;
using System.Threading.Tasks;
using System.Collections.Immutable;

Console.WriteLine("start");
await M(ImmutableArray.Create("a", "b", "c"));
Console.WriteLine("finish");

public partial class Program
{
    public static void M1(bool b) { }

    public static async Task M(ImmutableArray<string> a)
    {
        foreach (var i in a)
        {
            M1(i.Length is 10 or 20 or 30);
            await Task.FromResult(true).ConfigureAwait(false);
        }
    }
}

I am thinking we should just go ahead and merge the revert, take the new toolset when available (hopefully verifying that things work when we do that), and keep these notes available to revisit bringing the optimization back in the future.

@alrz
Copy link
Member

alrz commented Aug 18, 2023

I'm still not able to get a minimal reproducer. A program like the following doesn't NRE in either Debug or Release mode

I think it's because FromResult is already completed. I can get an NRE using Task.Delay(1000) instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area-Infrastructure untriaged Issues and PRs which have not yet been triaged by a lead

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants