Skip to content

Enhance run/rerun with the ability to record containers required for execution #2158

@mih

Description

@mih

It is hard to autodetect if/which containers are required for the execution of an arbitrary script via run. We could nevertheless record such info, by having an option, such as run --env singularity/conversion_tools. This could be given multiple times (if multiple containers are needed) and would merely result in a run-record in the commit message that captures this information. With such info we could then allow rerun to obtain the necessary containers prior execution. Containers would be linked as subdatasets, hence there is no need to capture a version separately.

As discussed in #2148, the base location for containers should be configurable (i.e. datalad.runtime.container-dir). Using such a config layer, will also enable later restructuring of a dataset, without breaking rerun be invalidating run commit records.

@kyleam What do you think?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions