Skip to content

Building same project in parallel can result in errors #2730

@mgsloan

Description

@mgsloan

I got errors like the following when running stack build in parallel with stack build --stack-yaml stack-8.0.yaml

/home/mgsloan/fpco/stack/src/main/Main.hs:96:1:
    Bad interface file: /home/mgsloan/fpco/stack/.stack-work/dist/x86_64-linux/Cabal-1.24.0.0/build/Stack/Types/Compiler.hi
        mismatched interface file versions (wanted "7103", got "8001")

This is due to the dist directory being shared. We could have lock files on the dist directory. However, it would be better if we allowed parallel builds of different configurations. So how do we vary the dist dir location for different configurations? One tricky constraint is that stack path --dist-dir should still work (possibly with a --stack-yaml command). There are two decent ways I can think of doing this:

  1. Hash the configuration, and use a trimmed version of it in the dist path. At first this approach seems promising, in fact possibly even handling CLI invocations that do things like modify the resolver. However, these CLI modifications would also need to be passed into stack path --dist-dir, which would be rather unwieldy.

  2. Add a new stack.yaml option config-name: which specifies a name for the configuration, along with a CLI option --config-name. If config-name is absent, it is inferred from the configuration filename by trimming off .yml or .yaml. This name is then used as a portion of the dist-dir path.

Either way, we'd also want to have lock files on the dist dirs, since we do want to prevent simultaneous execution on the same dist dir.

I'm favoring approach (2), because it would allow for a common case to work well without even knowing about config-name. You can have multiple stack configurations in different files, and safely build them in parallel.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions