Add Win11 helix-matrix/quarantine queue, remove win7/8 #36040

wtgodbe · 2021-09-01T17:53:58Z

wtgodbe · 2021-09-01T17:55:05Z

Queued https://dev.azure.com/dnceng/public/_build/results?buildId=1334918&view=results

HaoK

Looks good assuming the run doesn't crash and burn :)

HaoK · 2021-09-01T18:02:46Z

Usually the only thing to watch out for when adding new queues is we have some tests that are skipped on specific queues by name, which will fail whenever we add new ones, but generally those are linux / macOS

Tratcher · 2021-09-01T18:02:53Z

Have you run the HTTP/3 & QUIC tests locally recently? I'd expect some failures caused by recent runtime updates that we haven't addressed. I think @JamesNK has a few fixes pending in his PRs.

dougbu

Don't we have some Linux changes e.g. moving to a later version of Debian❔ Suggest we clean up the whole mess at once but it's your call.

wtgodbe · 2021-09-01T18:04:46Z

Have you run the tests locally recently?

Not all of them, but for now they're only running in the quarantine job so that shouldn't be blocking (unless there are unquarantined Quic tests?)

wtgodbe · 2021-09-01T21:10:35Z

@premun @MattGal getting an error with the new queue:

Sending Job to Helix-Client-Enterprise-Win11-Preview-08-30-2021...
D:\workspace_work\1\s.packages\microsoft.dotnet.helix.sdk\6.0.0-beta.21427.6\tools\Microsoft.DotNet.Helix.Sdk.MonoQueue.targets(55,5): error : ArgumentException: Unknown QueueId. Check that authentication is used and user is in correct groups. [D:\workspace_work\1\s\eng\helix\helix.proj]

MattGal · 2021-09-01T21:20:58Z

@premun @MattGal getting an error with the new queue:

Sending Job to Helix-Client-Enterprise-Win11-Preview-08-30-2021...
D:\workspace_work\1\s.packages\microsoft.dotnet.helix.sdk\6.0.0-beta.21427.6\tools\Microsoft.DotNet.Helix.Sdk.MonoQueue.targets(55,5): error : ArgumentException: Unknown QueueId. Check that authentication is used and user is in correct groups. [D:\workspace_work\1\s\eng\helix\helix.proj]

Taking a peek (this is the wrong queue name I'm just fetching it for you)

MattGal · 2021-09-01T21:23:15Z

@wtgodbe can you please try Windows.11.Amd64.ClientPre.Open? (it exists)

eng/targets/Helix.Common.props

wtgodbe · 2021-09-01T21:30:13Z

https://dev.azure.com/dnceng/public/_build/results?buildId=1335689&view=results

wtgodbe · 2021-09-02T16:13:51Z

IIS Cert failures on Win11:

"PS: Running 'C:\h\w\BCEE0A09\w\A66008D4\e\UpdateIISExpressCertificate.ps1' "

Major Minor Build Revision

10 0 22000 65
Certificate "github.com/aspnet/AspNetCore IIS testing cert" added to store.

CertUtil: -importPFX command completed successfully.

SSL Certificate deletion failed, Error: 2
The system cannot find the file specified.

wtgodbe · 2021-09-02T16:17:33Z

This is the same as we were seeing when we introduced #34726 (CC @Tratcher @MattGal), which is surprising as the queue is Win11. Unless the IIS Test Cert script is called before sending the job to helix, which certainly seems possible as it's part of HelixPreCommand

wtgodbe · 2021-09-02T16:48:20Z

Also the block I modified apparently doesn't effect quarantine-pr jobs, only the nightly quarantine runs, which is probably fine for getting coverage for now. Test job: https://dev.azure.com/dnceng/public/_build/results?buildId=1338195&view=results

wtgodbe · 2021-09-02T17:29:32Z

The IIS thing is the same that we saw when the Win10 pool we used got upgraded: https://github.com/dotnet/core-eng/issues/13582#issuecomment-886854042. The solution there was "please revert the upgrade", but I guess here we need to skip all of these tests on Win11 since it is necessarily "too new" - does that sound right @Tratcher?

Tratcher · 2021-09-02T20:23:23Z

The issue is in the project setup scripts, you can't even skip the tests, you'd have to skip whole projects.

Can you try to repro this locally so we can figure out why the script is failing? Is IIS Express not installed on this agent?

Tratcher · 2021-09-02T20:27:44Z

Looking at the script again, it seems to be this line failing?

aspnetcore/src/Servers/IIS/tools/UpdateIISExpressCertificate.ps1

Line 22 in cc5f896

Add-Content -Path $tempFile "http delete sslcert ipport=0.0.0.0:$i"

That would be expected to fail if IIS Express wasn't installed.

Also note this isn't exactly the same error as before, before it failed on add, not delete.

wtgodbe · 2021-09-02T20:38:36Z

Yeah, it works for me locally on Win11 (when ran as admin). @premun @MattGal is IIS Express not installed on the Win11 machines?

MattGal · 2021-09-02T20:50:30Z

Yeah, it works for me locally on Win11 (when ran as admin). @premun @MattGal is IIS Express not installed on the Win11 machines?

Checking out the definition I see only Windows10Base so unless it were preinstalled on the OS there's definitely no IISExpress here.

Checking the original issue (https://github.com/dotnet/core-eng/issues/13582) I don't see that being asked for... feel free to create a dotnet/core-eng issue with your list of requirements and we'll get onto it.

wtgodbe · 2021-09-02T20:54:29Z

It should be the same requirements as the Win10 machines where we run our IIS tests today, @Tratcher do you remember if we had to get specially configured machines for those?

Tratcher · 2021-09-02T20:57:24Z

Right, same requirements as the other windows agents.

The only extra thing we'll consider adding later is the Http3Enabled reg key.

wtgodbe · 2021-09-02T21:02:00Z

Looks like the 20h1 and 21h1 images enable IIS explicitly, but the Windows10-base image doesn't, so we'd want it explicitly enabled for the 11 image as well. I filed https://github.com/dotnet/core-eng/issues/14275 to track that work

Tratcher · 2021-09-02T21:03:00Z

IIS and IIS Express are separate. IIS Express is installed by VS, or as a standalone app.

wtgodbe · 2021-09-02T21:04:20Z

I misspoke, the 20h1 and 21h1 images install IIS Express: https://dnceng.visualstudio.com/internal/_git/dotnet-helix-machines?path=%2Fdefinitions%2Fshared%2Fwindows.yaml&version=GBproduction&line=982&lineEnd=983&lineStartColumn=1&lineEndColumn=1&lineStyle=plain&_a=contents

MattGal · 2021-09-02T21:12:20Z

I misspoke, the 20h1 and 20h2 images install IIS Express: https://dnceng.visualstudio.com/internal/_git/dotnet-helix-machines?path=%2Fdefinitions%2Fshared%2Fwindows.yaml&version=GBproduction&line=982&lineEnd=983&lineStartColumn=1&lineEndColumn=1&lineStyle=plain&_a=contents

Understood, please review the artifacts there if you can and let me know if there are any other differences you depend on.

wtgodbe · 2021-09-02T21:15:51Z

I think that's the only addition we need, I don't believe we need windows-enable-tls13 since it's already on by default in Win11

wtgodbe · 2021-09-08T17:34:30Z

Update is now rolled out

wtgodbe · 2021-09-08T17:34:36Z

/azp run

azure-pipelines · 2021-09-08T17:35:01Z

Azure Pipelines successfully started running 3 pipeline(s).

wtgodbe · 2021-09-08T17:35:11Z

And here's helix-matrix: https://dev.azure.com/dnceng/public/_build/results?buildId=1349415&view=results

wtgodbe · 2021-09-08T21:01:29Z

For quarantine validation: https://dev.azure.com/dnceng/public/_build/results?buildId=1350161&view=results

wtgodbe · 2021-09-08T23:31:14Z

Tests are running now, but we've got some Http3 failures on Win11 (CC @JamesNK @Tratcher)

Interop.FunctionalTests.Http3.Http3RequestTests.POST_ServerAbort_ClientReceivesAbort(protocol: Http3)

Assert.Equal() Failure
Expected: RequestCancelled
Actual: InternalError
at Interop.FunctionalTests.Http3.Http3RequestTests.POST_ServerAbort_ClientReceivesAbort(HttpProtocols protocol) in /_/src/Servers/Kestrel/test/Interop.FunctionalTests/Http3/Http3RequestTests.cs:line 441

Interop.FunctionalTests.Http3.Http3RequestTests.GET_GracefulServerShutdown_AbortRequestsAfterHostTimeout(protocol: Http2)

Assert.Equal() Failure
↓ (pos 4)
Expected: The connection was aborted because the server···
Actual: The HTTP/2 connection faulted.
↑ (pos 4)
at Interop.FunctionalTests.Http3.Http3RequestTests.GET_GracefulServerShutdown_AbortRequestsAfterHostTimeout(HttpProtocols protocol) in /_/src/Servers/Kestrel/test/Interop.FunctionalTests/Http3/Http3RequestTests.cs:line 1409

Interop.FunctionalTests.Http3.Http3RequestTests.GET_ServerAbort_ClientReceivesAbort(protocol: Http3)

Assert.Equal() Failure
Expected: RequestCancelled
Actual: InternalError
at Interop.FunctionalTests.Http3.Http3RequestTests.GET_ServerAbort_ClientReceivesAbort(HttpProtocols protocol) in /_/src/Servers/Kestrel/test/Interop.FunctionalTests/Http3/Http3RequestTests.cs:line 498

Interop.FunctionalTests.Http3.Http3TlsTests.ClientCertificate_AllowOrRequire_Available_Invalid_Refused(mode: AllowCertificate & mode: RequireCertificate)

Assert.IsType() Failure
Expected: System.Net.Quic.QuicOperationAbortedException
Actual: (null)
at Interop.FunctionalTests.Http3.Http3TlsTests.ClientCertificate_AllowOrRequire_Available_Invalid_Refused(ClientCertificateMode mode) in /_/src/Servers/Kestrel/test/Interop.FunctionalTests/Http3/Http3TlsTests.cs:line 189

wtgodbe · 2021-09-09T17:10:43Z

@JamesNK @Tratcher @adityamandaleeka any concerns with this PR? If not I can merge, backport, and open issues for the failing tests.

Tratcher · 2021-09-09T20:04:42Z

I'm concerned that none of those failures repro locally, but they're all quarantined right?

wtgodbe · 2021-09-09T22:09:32Z

/backport to release/6.0

github-actions · 2021-09-09T22:09:47Z

Started backporting to release/6.0: https://github.com/dotnet/aspnetcore/actions/runs/1219112590

Add Win11 helix-matrix/quarantine queue, remove win7/8

5d5101f

wtgodbe requested a review from a team as a code owner September 1, 2021 17:53

ghost added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Sep 1, 2021

Add 6.0 config

dec9acd

HaoK approved these changes Sep 1, 2021

View reviewed changes

dougbu approved these changes Sep 1, 2021

View reviewed changes

Tratcher reviewed Sep 1, 2021

View reviewed changes

eng/targets/Helix.Common.props Outdated Show resolved Hide resolved

Fix Win

ec9f2dc

Tratcher approved these changes Sep 9, 2021

View reviewed changes

wtgodbe merged commit 313f8db into main Sep 9, 2021

wtgodbe deleted the wtgodbe/Helix11 branch September 9, 2021 22:09

ghost added this to the 7.0-preview1 milestone Sep 9, 2021

github-actions bot mentioned this pull request Sep 9, 2021

[release/6.0] Add Win11 helix-matrix/quarantine queue, remove win7/8 #36349

Merged

Add Win11 helix-matrix/quarantine queue, remove win7/8 #36040

Add Win11 helix-matrix/quarantine queue, remove win7/8 #36040

Uh oh!

Conversation

wtgodbe commented Sep 1, 2021

Uh oh!

wtgodbe commented Sep 1, 2021

Uh oh!

HaoK left a comment

Choose a reason for hiding this comment

Uh oh!

HaoK commented Sep 1, 2021

Uh oh!

Tratcher commented Sep 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dougbu left a comment

Choose a reason for hiding this comment

Uh oh!

wtgodbe commented Sep 1, 2021

Uh oh!

wtgodbe commented Sep 1, 2021

Uh oh!

MattGal commented Sep 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MattGal commented Sep 1, 2021

Uh oh!

Uh oh!

wtgodbe commented Sep 1, 2021

Uh oh!

wtgodbe commented Sep 2, 2021

Uh oh!

wtgodbe commented Sep 2, 2021

Uh oh!

wtgodbe commented Sep 2, 2021

Uh oh!

wtgodbe commented Sep 2, 2021

Uh oh!

Tratcher commented Sep 2, 2021

Uh oh!

Tratcher commented Sep 2, 2021

Uh oh!

wtgodbe commented Sep 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MattGal commented Sep 2, 2021

Uh oh!

wtgodbe commented Sep 2, 2021

Uh oh!

Tratcher commented Sep 2, 2021

Uh oh!

wtgodbe commented Sep 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Tratcher commented Sep 2, 2021

Uh oh!

wtgodbe commented Sep 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MattGal commented Sep 2, 2021

Uh oh!

wtgodbe commented Sep 2, 2021

Uh oh!

wtgodbe commented Sep 8, 2021

Uh oh!

wtgodbe commented Sep 8, 2021

Uh oh!

azure-pipelines bot commented Sep 8, 2021

Uh oh!

wtgodbe commented Sep 8, 2021

Uh oh!

wtgodbe commented Sep 8, 2021

Uh oh!

wtgodbe commented Sep 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wtgodbe commented Sep 9, 2021

Uh oh!

Tratcher commented Sep 9, 2021

Tratcher commented Sep 1, 2021 •

edited

Loading

MattGal commented Sep 1, 2021 •

edited

Loading

wtgodbe commented Sep 2, 2021 •

edited

Loading

wtgodbe commented Sep 2, 2021 •

edited

Loading

wtgodbe commented Sep 2, 2021 •

edited

Loading

wtgodbe commented Sep 8, 2021 •

edited

Loading