This didn't work for the IoT repo. dotnet/iot#855 We are now getting another error:

SymStore.targets(22,41): error MSB4006: There is a circular dependency in the target dependency graph involving target "Build". [D:\a\1\s\build.proj]

Hmm :(

I think I see the problem. In IoT repo, Build depends on Pack, but with SymStore.targets DeployToSymStore target depends on _InnerDeployToSymStore which now also depends on Build, so it creates a cycle

consolidated repo has the same problem.

I don't know why it didn't repro locally to me after this fix. Maybe we do something special on CI. Looking now.

Edit: sorry about that it seems I missed the -ci flag locally and SymStore.targets only run in CI :(

@joperezr Yes, this is unexpected: "Build depends on Pack". Why do you need to have your custom build.proj? This is certainly not supported by Arcade SDK.

I think I can fix SymStore.targets to work regardless but I would strongly recommend removing the target customization from build.proj

We do something similar in the runtime repo:

As we redirect the target to Packages.builds which is the project that builds all our pkgproj files to generate packages.

https://github.com/dotnet/runtime/blob/master/src/libraries/build.proj#L70

Why do you need to have your custom build.proj? This is certainly not supported by Arcade SDK.

This is the way we have had it from the beginning and it is the same way that corefx has had it from the beginning as well. I realize that arcade consumption ideally would only have a solution at the root and have that drive the build, but both in corefx and iot this was not ideal as we have many projects which would make it a very large solution to load, when most of the time devs only need to focus on a single project. For this reason, with the help of the engineering team decided that having a build.proj that would orchestrate the build would be ideal, so saying that this is not supported by Arcade kind of surprises me to be honest.

https://github.com/dotnet/iot/blob/f92c6fcf4dcedaa5911c5ab8b0d3ff6ad0856881/build.proj#L70-L77

You can certainly have multiple solutions if you feel like you have too many projects in a single solution. For example, Roslyn has 2 solutions - one for compiler devs who are only interested in working on compiler layer and the other one that contains all 180 or so projects, which is used by IDE devs.

IOT does not have that many projects. VS should be able to handle that many projects just fine with good performance. If that's not the case and you experience slow VS please file bugs.

Another option, if you don't want to maintain a big solution is to use dir traversal to include all projects in the repo in eng\Build.props. See https://github.com/dotnet/arcade/blob/master/Documentation/ArcadeSdk.md#engbuildprops

IOT does not have that many projects.

In IoT we have a section under src/devices where we keep a catalog of device bindings which are community maintained. We have around 71 now (each has its own projects) but we expect this catalog to keep growing into the hundreds/thousands, and each binding has nothing to do with the other so it makes no sense to have both of them loaded on the same solution. This means that if we went with your approach, we would have to have hundred of solutions as well which won't scale.

In that case you can still use eng\Build.props customization.

That is what we do 😄 https://github.com/dotnet/iot/blob/master/eng/Build.props

Except that we need an orchestrator because there are projects that need to build before others, and we have to separate it with stages that run in order and that can be turned off, which is very similar to corefx build. Because of this need of the extra orchestrator, we had to use a build.proj. If Arcade supports an orchestrator that will let us select which projects to build in which order and when is it possible to paralelize, then we'll gladly move to that model.

cc @ericstj

If Arcade supports an orchestrator that will let us select which projects to build in which order and when is it possible to paralelize, then we'll gladly move to that model.

I see. Have you filed an issue on Arcade for this? It would certainly be better if we had first class support for this kind of thing.

Question though - why not express the dependency order via standard msbuild ProjectReference?

I haven't, but I will log a feature request for this so that corefx, iot and other repos can move off custom logic.

Question though - why not express the dependency order via standard msbuild ProjectReference?

IoT is a bit difficult to explain since we do custom RID cross-targeting, but in corefx for example, you want to have the first stage of the build to build all of the ref projects in order to construct a Targeting Ref pack, and then you want to have a second stage build which is the implementation and you want this to only have <Reference Include items which will automatically resolve into the right targeting pack. This logic is represented here: https://github.com/dotnet/corefx/blob/e99ec129cfd594d53f4390bf97d1d736cff6f860/src/dirs.proj#L5-L17. Note the SerializeProjects setting to make sure that these stages run in order.

Fix: #4405

Yes, I am aware that CoreFX (or rather dotnet/runtime at this point) is special, but I think that should be the only repo doing these kind of things.

I'm surprised IoT would need to use these tricks (like custom RID cross-targeting). Do we expect customers who build their IoT projects to not use the standard build targets we are shipping?

Do we expect customers who build their IoT projects to not use the standard build targets we are shipping?

Currently we are cross compiling for Windows/Linux so we do have custom targets defined in our project. That said, the plan is to remove them, and instead only have one build in the future with simple runtime decisions that will pick which implementation to use. This will help simplify our build, remove custom targets, and use the shipping targets but it will require some re-architecture work done on the design of the repo which we have scheduled for vNext.

@joperezr Excellent. Then I think we might not want to add extra complexity to Arcade right now. I'm fine with keeping the build.proj as is if there is indeed a plan to standardize the build.

I haven't, but I will log a feature request for this so that corefx, iot and other repos can move off custom logic.

@joperezr YES please :)