GitHub Community
-
Current behavior
Desired behavior
Sorry if this was discussed before I tried to search but didn't find anything. Our current implementationTo give it more context:
Imagine this scenario: Suggested solutionsone of the below would work
|
Beta Was this translation helpful? Give feedback.
Replies: 75 comments 52 replies
-
|
Hi @Shereef - thanks much for the feedback. Our initial design was focused on deployment scenarios, but we want to address the use case that you've outlined. I appreciate you outlining this so nicely and providing these suggestions. I hope that this is an area that we can address in the future. Thanks again. |
Beta Was this translation helpful? Give feedback.
-
|
Any updates on this? |
Beta Was this translation helpful? Give feedback.
-
|
@chrispat @tuves |
Beta Was this translation helpful? Give feedback.
-
|
@chrispat @tuves Once again asking if anyone is following up on this feature since ethomson left Github? |
Beta Was this translation helpful? Give feedback.
-
|
@ankneis could you see if anyone has anyone taken this on since ethomson left Github please? |
Beta Was this translation helpful? Give feedback.
-
|
2026: ➕ 1, or 💯 maybe I'm also at a loss for how to at least work around this shortcoming: multiple workflows, incl. unit tests etc, depend on having an up-to-date All these turnstile etc workaround ideas proved to simply not work out. Now a bunch of Help? |
Beta Was this translation helpful? Give feedback.
-
|
The desired behavior here, I believe, would be for jobs with the same |
Beta Was this translation helpful? Give feedback.
-
|
The OP is asking for jobs/workflows to wait and execute sequentially instead of canceling. Canceling existing workflows when a new workflow is triggered is already supported:
If your issue is specific to manually rerunning the same workflow, seems like a separate issue. |
Beta Was this translation helpful? Give feedback.
-
|
The said behavior is problematic as it's canceling it's own jobs in the same run. I have 6 jobs to run in each commit and they cancel each other in the same run. |
Beta Was this translation helpful? Give feedback.
-
|
Can your issue be solved by using concurrency on the workflow level instead of the job level? Afaik concurrency on the job level is meant more for branching paths (like their fallback value example). Sorry if I'm misunderstanding your usecase though |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
I think I just need the same, but add to a queue instead of cancel the workflow. I meant, don't run un parallel but in serial instead This will help, for example, use the same build result or cache in multiple runs for the same commit. The first one will create the artifacts/cache and the next one will be faster |
Beta Was this translation helpful? Give feedback.
-
|
That will complicate things even further. |
Beta Was this translation helpful? Give feedback.
-
|
Cache management is orthogonal to concurrency. Cache management across multiple jobs is something the workflow writer has to think about. Controlled concurrency among jobs in the same concurrency group, which is what this feature request is about, is not possible with what GHA provides out of the box. |
Beta Was this translation helpful? Give feedback.
-
|
Was surprised to see that canceling previous runs was the only option officially supported -- waiting and executing sequentially is what I need and seems to me like a clear usecase (canceling a deployment could result in a broken state) So far I've found this Action: https://github.com/ahmadnassri/action-workflow-queue But ideally, workflows sitting waiting for previous workflows to finish wouldn't consume action minutes in an official feature. |
Beta Was this translation helpful? Give feedback.
-
|
What I needed was exactly what https://github.com/styfle/cancel-workflow-action provided |
Beta Was this translation helpful? Give feedback.
-
|
We implemented something similar with a It's disappointing that the choice is between scaling up our self-hosted runners or surpassing our build minutes quota for github-hosted runners or building a separate queue outside of github actions to handle this kind of functionality. |
Beta Was this translation helpful? Give feedback.
-
|
We have a long running workflow that goes through several jobs to deploy AWS infrastructure. If one of them was cancelled mid-way, or deploys happened on to of one another, this would cause some pretty serious issues. We definitely need a way to queue jobs in sequence without consuming action minutes |
Beta Was this translation helpful? Give feedback.
-
|
Brainstorming - Eventually I will probably have to implement a specific self-hosted group. Maybe like 25 runners on a t3.small. Their job would be to handle queues. So I'd have a workflow or job dedicated to queing, and my larger runners (c7a.xlarge) wouldn't have to sit and wait for 15 mins at a time - instead I can have a single t3.small for all repos. |
Beta Was this translation helpful? Give feedback.
-
|
@justin-pierce the action-workflow-queue has an issue when there are multiple workflows started in the same second. See #45. @koreyGambill exactly, but do you know about some simple yet reliable queuing solution for that? |
Beta Was this translation helpful? Give feedback.
-
|
Would be great to be able to queue workflow runs (ideally without consuming action minutes). Was surprised that new runs cancel existing ones by default. |
Beta Was this translation helpful? Give feedback.
-
|
We hit this limitation while testing out the Merge Queue feature that is currently in limited beta. Our use case is ensuring that only one workflow is running for the PR at the top of the merge queue. When concurrency is set at the workflow level (for the merge_group event type) and multiple PRs are queued within the merge queue, the system cancels queued check suites once a new PR is added to the queue. It would be fantastic if we had the ability to disable canceling queued jobs! |
Beta Was this translation helpful? Give feedback.
-
|
If you're using self-hosted runners, a workaround/hack we used was to label one of these runners specifically, and then make all the jobs that need to be in a concurrency group just run on this labelled runner instead. No concurrency shenanigans. That causes them to queue up globally across all workflows without any caps on number of waiting jobs. The only major limitation is that if this runner fails, these jobs won't use any other runner. That was considered an acceptable risk to unblock us until we had a better solution. |
Beta Was this translation helpful? Give feedback.
-
|
Well, this is possible only for cases, where you need to serialize only a single job. |
Beta Was this translation helpful? Give feedback.
-
|
This is exactly our use case. |
Beta Was this translation helpful? Give feedback.
-
|
My team also needs this feature. |
Beta Was this translation helpful? Give feedback.
-
|
My team also needs this. |
Beta Was this translation helpful? Give feedback.
-
|
My team also need this feature - mainly for queueing mulitple actions each with Terraform plan/deploy steps which cannot be run at same time due to a lock on terraform state. |
Beta Was this translation helpful? Give feedback.
-
|
A must have for my team |
Beta Was this translation helpful? Give feedback.
-
|
Really hard to run terrafrom/terragrunt at scale without it |
Beta Was this translation helpful? Give feedback.
-
|
Really needing this to be implemented. I am currently using https://github.com/ben-z/gh-action-mutex for now, but this does pollute the git history. |
Beta Was this translation helpful? Give feedback.
-
|
Need this feature too. |
Beta Was this translation helpful? Give feedback.
-
|
GitHub, I'll add my team's need for this feature as well. I see a LONG list of replies like this, with no response from GitHub. (I'm in DevOps.) I'm going to put in a GitHub support ticket referencing this request to see if there are any plans to implement it. Our use cases is that I have created a highly parallelized building and deploying four distinct services in a monorepo. In this workflow, the update_service action is called by the jobs for each of those services, AND we have a very active CI/CD environment which can be processing several merges almost simultaneously. While I am discussing the use of the current concurrency model (cancel all pending workflow runs except the latest), until and unless that gets the green light our workflow runs will continue to risk colliding and corrupting the shared deployment environment with competing service updates. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the feature request, we need this feature |
Beta Was this translation helpful? Give feedback.
This comment was marked as off-topic.
This comment was marked as off-topic.
-
|
I voted on the issue, but the flood of plus one comments seems pretty impressive too. So, "hey, I want multiple overlapping jobs to queue and run serially instead of terminating" as well. I'm pretty sure this is possible, because one job will queue and multiple jobs waiting on runners will queue. So all the necessary logic seems like it's essentially present somewhere. 😅 |
Beta Was this translation helpful? Give feedback.
-
|
The more I try use GitHub actions for advanced use cases the more I stumble onto community conversations like this.
Yet, years later and still... nothing... Not so much as a reply with whether the feature is planned, on the roadmap at all or if this is something we should just learn to live with. |
Beta Was this translation helpful? Give feedback.
-
|
Actually, in this case, the silence is a very clear answer. It looks to me like Actions isn't the first class citizen, and it's not planned to change the state any soon. |
Beta Was this translation helpful? Give feedback.
-
|
+1 |
Beta Was this translation helpful? Give feedback.
-
|
Stop it. This thread don't need the spam that |
Beta Was this translation helpful? Give feedback.
-
|
@jeliasson is correct, please remember to use the upvote button or an emoji instead of creating a new thread to agree with a comment. It keeps the discussion clean. Thank You community!!! |
Beta Was this translation helpful? Give feedback.
-
|
Hey, sorry the Actions team has been quiet on this. We have added this to the list of paper cuts we are hoping to address in the next 12 months :) I don't have a timeframe for this one, but I want to say we know it's a pain point and do have it on our list <3 |
Beta Was this translation helpful? Give feedback.
-
|
Sadly that reply is meaningless vapor without any action whatsoever. |
Beta Was this translation helpful? Give feedback.
-
|
Good to see a reply, @nebuk89 . This is really a LOT more than a paper cut; it's a fact that many workflows are incrementally changing/updating things like databases, and MUST run in sequence. The current design presumes that later workflows are just an addition to the same codebase, and so are expected to produce the same result whether or not an intervening run is cancelled. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for keeping an eye and replying. Sorry I don't have solid updates yet. So thank you for taking the time <3 please keep engaging and never stop being passionate about us making Actions the best product it can be. |
Beta Was this translation helpful? Give feedback.
-
|
@nebuk89 If it helps for prioritization, this problem has caused correctness or production availability problems in different positions/companies when using GH Actions and is probably the most painful part of using GH Actions out of the box for a large team. I don't think there is any other GH Actions, GH feature in general, or any other non-GH CI/CD software feature gap that has caused me so much pain, stress, and lost hours for something that is reasonably expected to be configurable in tooling. There is probably even more impact than what is ascribed here because large teams may not even know this is where they need to go to flag why their CI steps decided to skip unexpectedly, which even if it causes down time, may only come up as a problem intermittently. Additionally, users appear to talk about this problem across several different community discussions that have not yet been consolidated, each with hundreds of upvotes. Thanks for the transparency and letting us know the situation. |
Beta Was this translation helpful? Give feedback.
-
|
💬 Your Product Feedback Has Been Submitted 🎉 Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users. Here's what you can expect moving forward ⏩
Where to look to see what's shipping 👀
What you can do in the meantime 💻
As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities. Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐ |
Beta Was this translation helpful? Give feedback.
-
|
Hi all! I have a more complex desired behavior: We have a step that deploys with terraform. We want that if a running job already reached that step, the other ones should wait instead of killing it. Ex:
Deployment succeeded, version |
Beta Was this translation helpful? Give feedback.
-
|
@hschimpf what about merge queues ? For a TF deploy it seems better to me to run the full deploy for each commits |
Beta Was this translation helpful? Give feedback.
-
|
This is exactly what we are looking for, @hschimpf were you successful in implementing something similar? @iusildra I agree that merge queue can do this, especially if we tweak the merge limits, but it also means that all the changes are stuck on the merge branch rather than in main |
Beta Was this translation helpful? Give feedback.
-
|
Per original request: my team needs something like
Motivating ExampleI have a 'deploy' GitHub Workflow that takes 600 seconds to run a deployment.
Requirements
Expected Behavior
Documented Behavior
|
Beta Was this translation helpful? Give feedback.
-
|
I can understand why it is so difficult to implement. There are many bad jobs out there, which run when they should not. So current behavior prevents them from overwhelming Github servers. Especially if you have heavy jobs running after every commit. But at the same time there are people out there who are willing to throw money at you just for using your CPU. So this is clearly revenue possibility for you. So why it was not implemented already for paid clients, is funny mystery. For people who will cry , that this unfair - do you imagine Github allowing to queue jobs faster than they can execute for free? |
Beta Was this translation helpful? Give feedback.
-
|
That can be easily addressed. You can have explicitly defined queues and set limits on queue sizes and you can add rate limits. This would discard excessive runs of badly behaving jobs. You can implement something like this with what's already in place, but it's tedious and more resource consuming than if Github would do it. So it can't be that difficult. Only, it won't appear out of nothing. Somebody needs to implement it, and apparently MS isn't willing to pay for it. |
Beta Was this translation helpful? Give feedback.
-
|
There is a workaround if you have self-hosted runners.
my-job:
runs-on: [self-hosted, linux, cpp]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# ... etc. ...Runners themselves have no maximum queue length, so all the heavy jobs will wait their turn properly. Note that there is a 24 hour time limit for such queued jobs. |
Beta Was this translation helpful? Give feedback.
-
|
Hah, this is pretty brilliant. I may have to resort to this 🤦♂️ |
Beta Was this translation helpful? Give feedback.
-
|
Found that softprops/turnstyle helps to workaround this limitation. Instead of relying on jobs with concurrency groups to begin, first I run that action that holds it while there are other workflows still running. |
Beta Was this translation helpful? Give feedback.
-
|
It can work for cases when runners are free. And for huge compiling hosts, it's a money black hole. |
Beta Was this translation helpful? Give feedback.
-
|
We set it on a single action job "queue" that |
Beta Was this translation helpful? Give feedback.
-
|
Ok y'all 🤫 we have shipped but not publicly announced that we have shipped If anyone wants to try it out, have a go :D and let me know any initial thoughts 
|
Beta Was this translation helpful? Give feedback.
-
|
My recommendation in that case would be to separate the approval gate and the concurrency group into sequential jobs in the workflow. Then the concurrency job would only start waiting after the approval job completed. The caveat is that ordering would not be preserved: jobs would run in the order they were approved, which could be different from the order the workflows were originally dispatched. Will that work for you? |
Beta Was this translation helpful? Give feedback.
-
Our workflow has jobs A, B, C, D with dependencies (A, C), (B, C), (C, D). |
Beta Was this translation helpful? Give feedback.
-
|
I may be misunderstanding the way this is supposed to work, but using the new |
Beta Was this translation helpful? Give feedback.
-
|
@amfage, There's not enough information provided to confirm.. but from the screenshots it looks like you are using merge queues, which may group multiple PRs together and merge them at the same time as one merge commit. So that could explain why your on-push action is only triggered once when multiple PRs are merged. I think it's unrelated to use of |
Beta Was this translation helpful? Give feedback.
-
This works great for our purpose thanks!
|
Beta Was this translation helpful? Give feedback.
-
|
Awesome improvement. The only thing that's missing for our use case is an option to queue specific jobs however ensure execution order the same as the workflows where startet, all other jobs before should be able to run in any order because the don't have any interference. |
Beta Was this translation helpful? Give feedback.
and others
Ok y'all 🤫 we have shipped but not publicly announced that we have shipped
If anyone wants to try it out, have a go :D and let me know any initial thoughts