Skip to content

Requiring shared directory for remote execution of tasks #1535

@BoPeng

Description

@BoPeng

Currently the sos task execution system is complicated because it needs to copy files around if files reside on directories that are not shared by the job submission machine and the head node. A lot of work, such as signature checking, directory mapping, and file synchronization has been done to enable a powerful, yet complicated and confusing system that can easily break.

I propose that we remove all the file mapping feature and require file systems to be shared. In this way input and output files have to exist on both local and remote file systems, and there is no need to map directories and copy files around. The system will be much easier to understand, configure, and a lot more robust.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions