@safern @Anipik @joperezr I'm in the process of trying to migrate all remaining non-helix tests into helix. I'm going to keep going with the remaining test projects but I did packageinstall and packagevalidation already as part of my first pass. However, there are test failures.

Did ya'll want me to disable the tests and file an issue for you to follow up to fix or did you want to fix the tests in this PR?

For packagevalidation, I determined it's because one of the test project depends on the compatibility targets and project from source so I have that copying into the test publish directory. You just need to copy it into the test directory and modify the test assets to pick it up. The install failure I haven't looked into.

Additionally, the packagevalidation test project is building for 472 and 7.0 and running against both. Is 472 needed or can it be removed (we only have one set of tests that run on 472 because the sdk resolver runs on netfx in VS)?

Thanks a lot for doing this @marcpopMSFT!

Did ya'll want me to disable the tests and file an issue for you to follow up to fix or did you want to fix the tests in this PR?

Whatever works best for you, I suppose either me or @safern can probably work on this tomorrow and it sounds like the fix should be trivial, but don't be blocked by us, if you want to merge this in earlier, feel free to just log the issue against us and disable them. AFAIK there is no work in progress going into PackageValidation this week, so disabling the tests for a couple of days wouldn't be the end of the world (@safern please correct me if I'm wrong)

Additionally, the packagevalidation test project is building for 472 and 7.0 and running against both. Is 472 needed or can it be removed (we only have one set of tests that run on 472 because the sdk resolver runs on netfx in VS)?

Ideally we would want to keep the net472 tests going, as we do cross-compile for net472 and it is a major scenario for us since it is the coverage we have for building inside VS. So unless there is a good reason why not to keep running them in net472, I'd suggest to keep doing so.

Whatever works best for you, I suppose either me or @safern can probably work on this tomorrow and it sounds like the fix should be trivial, but don't be blocked by us, if you want to merge this in earlier, feel free to just log the issue against us and disable them. AFAIK there is no work in progress going into PackageValidation this week, so disabling the tests for a couple of days wouldn't be the end of the world (@safern please correct me if I'm wrong)

Yes we can disable the test and open an issue and either @joperezr or me can take care of it tomorrow or next week.

Thanks for doing this.

Ideally we would want to keep the net472 tests going, as we do cross-compile for net472 and it is a major scenario for us since it is the coverage we have for building inside VS. So unless there is a good reason why not to keep running them in net472, I'd suggest to keep doing so.

Adding to this this also provides us with coverage if someone uses full msbuild to pack (i.e MSBuild /t:pack).

Thanks all. I filed this issue to track the 6 failing package validation tests that all use the same test project and included some details as to what needs to get done. #23533

I also marked the packagevalidation test to be run for net472. We'll see how they work in helix as I'm still working on getting the msbuildsdkresolver tests to correctly run there as well.

Thanks, @marcpopMSFT! So far changes for PackageValidation tests look good.

@safern I'm still working on getting the package validation tests clean. I realized that there appears to be a first run issue. If you see the two pieplines below, two different packagevalidation tests failed with the same issue that the first run message was reported when it expected nothing.
https://dev.azure.com/dnceng/public/_build/results?buildId=1576856&view=ms.vss-test-web.build-test-results-tab
Expected collection to be empty, but found {"\r\nWelcome to .NET 7.0!\r\n

https://dev.azure.com/dnceng/public/_build/results?buildId=1565425&view=ms.vss-test-web.build-test-results-tab
Expected collection to be empty, but found {"\r\nWelcome to .NET 7.0!\r\n

This is failing at this check but that may just be because it's not expecting to see the first run message (maybe this has to do with running on a clean helix machine). Is there an initialization method in the packageinstall tests where a dotnet call can be added to skip past the first run message? Any other suggestions?
https://github.com/dotnet/sdk/blob/main/src/Tests/Microsoft.DotNet.PackageInstall.Tests/ToolPackageInstallerTests.cs#L843