At the moment we can run only 250 jobs per workflow at the same time because of the GitHub job matrix limitations. With this restriction processing of newer versions of libraries that we have in the repository is slow and we are processing 250 new versions on a bi-weekly base.
Idea: because the limitation applies on a workflow level, we can have:
- one workflow that knows the full list of libraries versions that we want to test, and orchestrates (spawns) other workflows to run the batches
- workflow that would accept batch of 250 jobs and execute them in parallel
This way, the orchestration workflow can dynamically spawn many worker workflows that will execute batches of tests (which will run in parallel).
At the moment we can run only 250 jobs per workflow at the same time because of the GitHub job matrix limitations. With this restriction processing of newer versions of libraries that we have in the repository is slow and we are processing 250 new versions on a bi-weekly base.
Idea: because the limitation applies on a workflow level, we can have:
This way, the orchestration workflow can dynamically spawn many worker workflows that will execute batches of tests (which will run in parallel).