Tested the concepts, but not the final PR in that constellation, yet.

Now tested. Forgot to pass through permission for the tag job, fixed.

It works. Here's how:

• When changing the title of a PR, we get a single skipped job:
  
  Because this is triggered via a different workflow file, the successful jobs from before are still visible and are not removed.
• When changing the base branch of the PR, we get additional jobs. Example:
  
  All the jobs that are new for the base branch change are prefixed with "Edited / Base /". In this example we can see that the old job from before are also not replaced, thus the eval from before is still running. We can't prevent them from still showing up, but we can eventually skip the jobs that are still running with something like ci/eval: cancel previous run if new code is pushed #357990 (comment).

Not visible on the second screenshot, yet: We are also running eval again after the base branch change, which is the point of all of this:

All of that happened in wolfgangwalther#632.

@infinisil WDYT?

I reviewed it for now. Clever! Though also rather messy 😅, so I'm a bit undecided whether we should have this.

It's the least messy, at least by output, that I could make it :D, but yes I agree.

• Doesn't run CI when PR title/body is changed.

  • Less resource usage
  • No more waiting for CI in such cases.

This happens quite frequently to me, when I'm merging backport PRs. Building and testing stuff, then going through a final check of the changelog.. yeah all, good set the checkmark "acceptable for release".. ah, damn CI is running again.

I merge in those cases and don't wait for CI, but I'll always get error notifications, because those jobs can't finish when the PR's branch is suddenly gone.

• Duplicated parts in the workflows

• More complexity

My idea was to come back with more of the same. In this case it's technically a "correctness" issue, so I did that first.

But I also think the following is quite annoying:

I have a ton of checks.. and those visible at the top are always a list of "get-merge-commit", because we run this so often.

With the same approach as in edited-base.yml, i.e. calling those other workflows via workflow_call, we could run the merge-commit job once and pass the result to all other workflows. This would give us much less cluttered output.

One more step would then be to collect most of the small "check" jobs into a single job, so it only appears once. I really don't need to see maintainers-sorted, nix-format, nixf-tidy, editorconfig etc. etc. all in separate jobs in the list. They are almost never failing and I could easily look it up which step failed.

My goal: Make the "check list" usable without scrolling. Not everything, but at least have useful information at the top already.

We could probably make this a bit easier to understand in the workflows folder, if we went a step further, made all actual jobs be called by workflow_call only. Then put them into a jobs folder or so, and have only those "trigger workflows" in the workflows folder.

Doing it this way would also give us a nice overview of: One file has all the jobs that will run on a branch push, one file has all the jobs that will run on a PR, ... Potentially easier to reach consistent results with that.

Based on that, and unless I'm missing anything, I don't feel inclined to have this.

Hopefully my vision above changes your mind ;)

Or.. maybe you have other ideas about the issues mentioned?

made all actual jobs be called by workflow_call only

I really like this idea. The selection logic being under workflow control seems like the only way to avoid fatigue over there being yet another check that I need to run locally prior to proposing a PR... and is directionally aligned with running CI locally, outside of GitHub, so we can replicate locally what it would do.

Marked as draft for now. I will come back to it, but will do other things first.

I changed this to a much simpler approach: Whenever a pull request is edited we run a custom workflow. This runs its only job only when the edited event was caused by a change of the base branch. If that's the case, the job closes and re-opens the PR. This will trigger an entirely new CI run.

Advantages:

• The knowledge about "base branch changed -> edited -> complications" is all central in one file only.
• Other workflows don't need to consider this at all, it's built-in for them, when they use the default pull_request_target.
• This only ever kicks in, when the base branch is really changed and leaves a minimum of traces otherwise (only single skipped job when editing the title of a PR etc.)
• No duplicated jobs as in the solution before.

Test run was here:
wolfgangwalther#638

This PR in it's current state solves two problems:

• Editing titles and descriptions of PRs currently trigger new jobs,
• Changing the base branch of a PR without pushing a rebase does not trigger a new Eval run

Compared to my previous implementation this is nicely isolated in a single file and not overly complex. The only slightly odd thing is, that when you change the base branch of a PR, CI will close and immediately open it again. Since changing the base branch is a rather rare event, compared to editing titles and descriptions, I think this compromise is well worth it.

Since there was no objection to this approach, I will go ahead with it.

Successfully created backport PR for release-25.05:

• [Backport release-25.05] workflows/eval: run when base branch changed #409857