Skip to content

Add support for swarm jobs#40307

Merged
AkihiroSuda merged 1 commit into
moby:masterfrom
dperny:swarm-jobs
Jan 20, 2020
Merged

Add support for swarm jobs#40307
AkihiroSuda merged 1 commit into
moby:masterfrom
dperny:swarm-jobs

Conversation

@dperny

@dperny dperny commented Dec 12, 2019

Copy link
Copy Markdown
Contributor

proposal (in swarmkit): moby/swarmkit#2852

- What I did

Adds support for Jobs in Swarm Mode

Jobs are Services which run some number of Tasks to completion (the container exiting with code 0) and then stop executing new Tasks.

- How I did it

  • Bumps swarm vendoring to the latest master, which includes all of the swarm-side jobs changes.
  • Adds two new service modes: ReplicatedJob and GlobalJob.
    • ReplicatedJob has two fields: MaxConcurrent, which sets the maximum number of tasks to run at the same time for the job, and TotalCompletions, which is the number of Completed tasks desired for the job. If MaxConcurrent is not set, it will default to 1. If TotalCompletions is not set, it will default to the value of MaxConcurrent
    • GlobalJob has no fields. Like a global service, global jobs are scheduled on every node in the cluster matching placement constraints.
  • Adds a new field to the Service object, present when the Service is a job: JobStatus, which includes a counter for running the same job multiple times.
  • Adds a new field to the Task object, present when it belongs to a job: JobIteration, which is identical to the value of JobStatus.JobIteration on the Service that spawned this Task
  • Updates swagger.yml with documentation for all new fields.
  • Updates version-history.md to include information about new jobs fields.

- How to verify it

Includes new integration tests for jobs functionality.

- Description for the changelog

Swarm Mode now supports running Jobs.

@dperny

dperny commented Dec 12, 2019

Copy link
Copy Markdown
Contributor Author

Mostly running this PR as WIP because my local dev box is giving me trouble, and I want to run against CI a bit to see if the problem is more general.

@dperny
dperny force-pushed the swarm-jobs branch 3 times, most recently from fff938e to bb94bda Compare December 13, 2019 18:25
@dperny dperny mentioned this pull request Dec 13, 2019
@thaJeztah thaJeztah added area/swarm impact/api impact/changelog kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny status/2-code-review labels Dec 27, 2019
@dperny
dperny force-pushed the swarm-jobs branch 5 times, most recently from c930baa to dbd0ef6 Compare January 8, 2020 16:36
@dperny

dperny commented Jan 9, 2020

Copy link
Copy Markdown
Contributor Author

It seems that in CI, the Jobs test takes too long to complete 20 iterations. I've cut down the total to 7, which is less than the minimum completion I saw, which was 8.

@thaJeztah

Copy link
Copy Markdown
Member

@derny this needs a rebase now

@dperny

dperny commented Jan 9, 2020

Copy link
Copy Markdown
Contributor Author

@thaJeztah Yes, it does, but in any case, the development on Jobs has happened all in a feature branch in Swarmkit, and that feature branch has not quite yet been merged into swarmkit master.

@dperny dperny changed the title WIP: Add support for swarm jobs Add support for swarm jobs Jan 13, 2020
@dperny

dperny commented Jan 13, 2020

Copy link
Copy Markdown
Contributor Author

No longer WIP.

@cpuguy83 cpuguy83 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when requested number of completions is hit and there are still others running?

Comment thread daemon/cluster/convert/service.go Outdated
Comment thread api/types/swarm/service.go Outdated
@dperny

dperny commented Jan 13, 2020

Copy link
Copy Markdown
Contributor Author

Swarmkit will only schedule the number of Tasks needed to reach TotalCompletions, so there won't be any Tasks still running when TotalCompletions is reached. MaxConcurrent establishes the upper bound on how many are scheduled.

Adds support for ReplicatedJob and GlobalJob service modes. These modes
allow running service which execute tasks that exit upon success,
instead of daemon-type tasks.

Signed-off-by: Drew Erny <[email protected]>

@cpuguy83 cpuguy83 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Nicely done!

@collabnix

Copy link
Copy Markdown

Would love to blog around this. Which Docker release is it coming with? Beta bits?

@dperny

dperny commented Jan 16, 2020

Copy link
Copy Markdown
Contributor Author

CLI PR is open, pending the merge of this PR.

@AkihiroSuda
AkihiroSuda merged commit f9d136b into moby:master Jan 20, 2020
@arseniybanayev

Copy link
Copy Markdown

Amazing. Thank you for the work! When is this planned to be released?

@BretFisher

Copy link
Copy Markdown

👏 👏 💯

@crazy-max

Copy link
Copy Markdown
Member

Very nice, thanks for this! I'll soon be able to archive swarm-cronjob 😅

@tsbx

tsbx commented Jan 20, 2020

Copy link
Copy Markdown

@dperny Thanks for this new feature 👏
@crazy-max When reading the documentation, it seems different from your (crazy 😄 ) swarm-cronjob project: here there is a limited number of executions (TotalCompletions) and no cron equivalent functionality, so please don't archive it. It's still very useful ❤️

@ivorscott

Copy link
Copy Markdown

@dperny you're a blessing to open source. thank you.

@Ohtar10

Ohtar10 commented Oct 21, 2020

Copy link
Copy Markdown

This is really good stuff!

Question: Would it be possible to specify jobs via compose/stack file too? or this will only work via CLI?

Edit: Moving the question to docker/cli#2262 which is where it was being discussed.

@alexellis

Copy link
Copy Markdown
Contributor

Nice work! I would have loved this three years ago 😄 I've actually seen a rise in interest in alexellis/jaas so I think this will be well received by the community of users still on Swarm.

@ghost

ghost commented Jul 31, 2023

Copy link
Copy Markdown

Is it possible to create cron job with swarm?

@mbrodala

Copy link
Copy Markdown

Answering the last question: evidently not. Swarm jobs are meant to run once and will not be run again if completed:

Jobs are never restarted on reaching the Complete state. This means that for jobs, setting --restart-condition to any is the same as setting it to on-failure.

Source: https://docs.docker.com/reference/cli/docker/service/create/#running-as-a-job

@thaJeztah

Copy link
Copy Markdown
Member

I may have written up some example somewhere, but I think you can somewhat mimic it with a restart policy and delay;

docker service create \
  --name cronish \
  --replicas 1 \
  --restart-condition any \
  --restart-delay 1h \
  alpine sh -c 'do-the-thing'

@mbrodala

Copy link
Copy Markdown

Yeah, read something like this on Reddit but this doesn't make use of the job feature at all.

(Also --restart-condition already defaults to any so that can be omitted.)

@olljanat

Copy link
Copy Markdown
Contributor

Yeah, read something like this on Reddit but this doesn't make use of the job feature at all.

True, jobs support in Swarm is limited by design. https://github.com/crazy-max/swarm-cronjob does what you are looking for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/swarm impact/api impact/changelog kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny status/2-code-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.