CI: figure out plans for lock file updates #23637

Activity

1. lucascolley

   added

   CIItems related to the CI tools such as CircleCI, GitHub Actions or AzureItems related to the CI tools such as CircleCI, GitHub Actions or Azure

   on Sep 18, 2025
2. lucascolley commented on Sep 21, 2025

   

   lucascolley

   on Sep 21, 2025

   MemberAuthor

   More actions

   https://github.com/modular/modular-community/blob/main/.github/workflows/update-lockfiles.yml
3. lucascolley commented on Sep 25, 2025

   

   lucascolley

   on Sep 25, 2025

   MemberAuthor

   More actions

   following gh-23658, we are now in line with the outline in the top-post there:

   The idea for pinning which motivates these changes:

   • pins will largely be enforced by the lock file (pixi.lock), not by the manifest (pixi.toml)

     • except for those maintained manually, like those to select a Python version, or those like:

   # JAX 0.6.2 and 0.7.0 segfault on CUDA
   jaxlib = { version = "!=0.6.2,!=0.7.0", build = "cuda12*" }

   • those pins will be refreshed infrequently in the main repo

   • something like every 3 weeks was suggested (to reduce churn and repo bloat)

   • this could be automated with something like modular/modular-community@main/.github/workflows/update-lockfiles.yml

   • a separate repo could automate more frequently refreshing those pins and running tests to smoke out problems with new releases of dependencies ahead of time

     • this would keep that noise away from the main repo, but make sure we can keep on top of our dependencies breaking things more steadily, rather than being hit with it all at once every 3 weeks
     • we could automate opening issues to the main repo with something like scikit-learn/scikit-learn@main/maint_tools/update_tracking_issue.py

   A few further thoughts past that:

   • @VeckoTheGecko pointed out that a separate repo seems superfluous: automated PRs seem unnecessary, all we may need is a cron job which can run on the main repo, calling update or upgrade from the manifest's state on main, and reporting feedback in an issue automatically
     • unless failures of this cron job would themselves ping people watching the repo? Could we turn that off if so?
   • we may want to replace the no-pin strategy with constrained versions in the manifest. I opened rfc: add no-upgrade field to opt-out of pixi upgrade in manifest prefix-dev/pixi#4644 to discuss what I think is needed to make automated upgrades nice, if we went down that route.

   👀React with 👀1Reacted by Nick Hodgskin
4. lucascolley commented on Oct 1, 2025

   

   lucascolley

   on Oct 1, 2025

   MemberAuthor

   More actions

   @VeckoTheGecko pointed out that a separate repo seems superfluous: automated PRs seem unnecessary, all we may need is a cron job which can run on the main repo, calling update or upgrade from the manifest's state on main, and reporting feedback in an issue automatically

   Ah, on second thought — while the update/upgrade part can indeed be done without creating a branch, running the various workflows we have on top of the updated lock file I think does require creating/updating a branch (at least in the simplest implementation).

   I started a branch to figure out what the boilerplate might look like for such an upgrade workflow at https://github.com/lucascolley/scipy/blob/autolock-iter/.github/workflows/smoke_dependency_upgrades.yml.

   👍React with 👍1Reacted by Nick Hodgskin
5. lucascolley commented on Oct 1, 2025

   

   lucascolley

   on Oct 1, 2025

   Last edited by lucascolley

   MemberAuthor

   More actions

   draft workflow for automated PRs for lock file updates: https://github.com/lucascolley/scipy/blob/lock-update/.github/workflows/update_lock_file.yml

   We probably want to wait until the lock file is extracted from .github/workflows so that GitHub doesn't complain about workflows meddling in the workflows directory

   👍React with 👍1Reacted by Ralf Gommers