Migration slot 45 diverges between release-1.5 and release-1.6, silently skipping the Kubeadm keep-policy migration #3534

What happens

Migration slot 45 now holds a different migration on release-1.5 than on release-1.6, so a cluster that installs the next 1.5 patch and then upgrades to 1.6 silently skips release-1.6's migration 45.

Slots 1 through 44 are semantically aligned between the two branches. Slot 45 diverged when #3370 was merged to release-1.5, which shipped a new migration at 45 and raised targetVersion from 45 to 46.

The mechanism

packages/core/platform/images/migrations/run-migrations.sh walks seq $CURRENT_VERSION $((TARGET_VERSION - 1)) and its only guard is that a migration file must be present in the image. There is no cross-branch guard, so a slot claimed by a maintenance branch is skipped on the forward path rather than reported. This is exactly the forward-upgrade trap documented in docs/release.md under "Backporting code that touches monotonic counters".

The exposed path is patch-then-minor, which is the normal upgrade order:

• A cluster on v1.5.2 is stamped cozystack-version=45.
• It installs the next 1.5 patch, runs release-1.5's migration 45, and is stamped 46.
• It upgrades to 1.6. The runner walks seq 46 53, so release-1.6's migration 45 never runs.

A cluster that goes from v1.5.2 straight to 1.6 is unaffected — it is stamped 45 and runs release-1.6's migration 45 normally.

Impact

release-1.6's migration 45 pins KubeadmConfigTemplate and KubeadmConfig with helm.sh/resource-policy=keep so the Talos worker chart change does not let Helm prune them while CAPI still references them from the existing kubeadm-backed MachineSet. Skipping the pin risks that prune on affected clusters.

Why #3370 should not be reverted

release-1.5's migration 45 is a data-loss repair, not a convenience. Migration 43 matched the owning release against the literal seaweedfs-system, so it only fired for an instance named seaweedfs; a user-created instance foo is owned by release foo-system and was skipped, never received helm.sh/resource-policy: keep, and is deleted as a removed resource when the post-split foo-system chart no longer renders the Cluster. CNPG takes the PVC with the Cluster, so the filer metadata — and with it every object in that tenant's S3 — becomes unreachable. It also breaks a re-prune loop that otherwise recurs on every upgrade attempt. So the migration has to run, which means targetVersion has to advance.

Suggested fix

Mitigation 2 from docs/release.md: add an idempotent duplicate of release-1.6's migration 45 at slot 54 on release-1.6 and raise its targetVersion from 54 to 55. A cluster arriving stamped 46 then walks seq 46 54 and picks the pin up at 54; a cluster already on 1.6 stamped 54 runs the duplicate once, which is a no-op because stamping an annotation is idempotent.

One thing to verify when implementing: the pin must land before tenant kubernetes HelmReleases converge to the 1.6 chart, since that convergence is what would prune the objects. release-1.6's migration 45 is an annotation pin that runs in the platform pre-upgrade window, so a duplicate at 54 should still precede tenant release convergence, but this is worth confirming on a real upgrade rather than assumed.

Also worth considering

Nothing reports this class of divergence today — it was found by hand while auditing backports. A cheap guard would be a CI check comparing migration slot alignment across the supported release branches, failing when a slot below the lowest targetVersion holds different migrations on two branches.