Skip to content

fix migrated cri image config when using registry#12617

Merged
dmcgowan merged 1 commit intocontainerd:mainfrom
akhilerm:fix-mirror-migration
Mar 6, 2026
Merged

fix migrated cri image config when using registry#12617
dmcgowan merged 1 commit intocontainerd:mainfrom
akhilerm:fix-mirror-migration

Conversation

@akhilerm
Copy link
Copy Markdown
Member

@akhilerm akhilerm commented Dec 3, 2025

when config has registry.mirrors from config version 2, after migration, config_path should be set to empty and not be the default value

move setting the config_path from default image config to plugin init.
This makes sure that when both mirrors and config_path are set, mirrors
are given preference and does not cause an error with CRI plugin

Fixes: #12612

Comment thread plugins/cri/images/plugin.go Outdated
Comment on lines +216 to +223
configPath, configPathExists := regMap["config_path"]
if _, ok := regMap["mirrors"]; ok {
// This is needed because if the config_path value is set along with mirrors, it should be made
// empty
if configPathExists {
regMap["config_path"] = ""
}
} else if configPathExists && configPath == "" {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making this change here does not actually work, because further later after the configMigration is done, the default values of config_path is used, which gets added when LoadPlugins is called

func LoadPlugins(ctx context.Context, config *srvconfig.Config) ([]plugin.Registration, error) {

@akhilerm
Copy link
Copy Markdown
Member Author

/cc @dmcgowan

@akhilerm akhilerm force-pushed the fix-mirror-migration branch from 752cce9 to b00ed38 Compare December 12, 2025 15:10
@akhilerm akhilerm marked this pull request as ready for review December 12, 2025 18:25
@dosubot dosubot Bot added the area/cri Container Runtime Interface (CRI) label Dec 12, 2025
Comment thread plugins/cri/images/plugin.go Outdated
// if mirrors are present, prefer using the mirrors and set the
// config_path to empty.
if len(config.Registry.Mirrors) > 0 {
config.Registry.ConfigPath = ""
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In previous releases, this would cause an error, right? I can understand some users may have migrated to a breaking config, but erroring is more consistent than silently clearing out a field. Is there another case this is trying to cover?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In previous releases, this would cause an error, right?

In 2.2 or 1.x series?

I can understand some users may have migrated to a breaking config
But that would have happened only after defaulting the config_path values right?

if we error out here, it will be same as validation, i.e cannot have both config path and mirrors set at the same time and error out.

@akhilerm akhilerm force-pushed the fix-mirror-migration branch 2 times, most recently from c8d977c to 8a84437 Compare December 18, 2025 16:44
Comment thread plugins/cri/images/plugin.go Outdated
// config path are empty.
// This also makes sure that if the loaded config already contains both config path and mirrors
// the validation will fail.
if config.Registry.ConfigPath == "" && len(config.Registry.Mirrors) == 0 {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmcgowan This should ideally solve the issue. also not clearing any values that the users might have already set.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the biggest downside of this is that containerd config default will no longer have the config_path value. But it's unclear to me how much anyone might depend on that, and the deleted code already did something similar so...I think this is probably fine for now.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tools like nvidia-container-toolkit uses the containerd config default command to create the config that is used as the base for the drop in config. AFAIK, they modify only the runtime key and not the images key. But will verify if config_path is used when some options are enabled.

@akhilerm akhilerm force-pushed the fix-mirror-migration branch from 8a84437 to db6e225 Compare February 18, 2026 03:43
@akhilerm akhilerm moved this from Needs Triage to Needs Reviewers in Pull Request Review Feb 19, 2026
@akhilerm akhilerm force-pushed the fix-mirror-migration branch from db6e225 to 78a41d8 Compare March 3, 2026 19:13
Copy link
Copy Markdown
Member

@chrishenzie chrishenzie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! We're currently hitting this bug internally, and it's blocking our adoption of containerd 2.2. Is there any pending feedback or anything we can do to help move this along?

@samuelkarp samuelkarp force-pushed the fix-mirror-migration branch from 78a41d8 to d58df07 Compare March 6, 2026 18:07
@samuelkarp
Copy link
Copy Markdown
Member

samuelkarp force-pushed the fix-mirror-migration branch from 78a41d8 to d58df07

Rebased on current HEAD of the main branch. No other changes.

@github-project-automation github-project-automation Bot moved this from Needs Reviewers to Review In Progress in Pull Request Review Mar 6, 2026
@samuelkarp samuelkarp requested a review from dmcgowan March 6, 2026 18:14
Comment thread plugins/cri/images/plugin.go Outdated
@github-project-automation github-project-automation Bot moved this from Review In Progress to Needs Update in Pull Request Review Mar 6, 2026
move setting the config_path from default image config to plugin init.
only set the default value when both mirrors and config_path are empty,
in all other cases retain the existing behaviour

Co-authored-by: Samuel Karp <[email protected]>
Signed-off-by: Akhil Mohan <[email protected]>
Signed-off-by: Samuel Karp <[email protected]>
@samuelkarp samuelkarp force-pushed the fix-mirror-migration branch from d58df07 to 1d77b68 Compare March 6, 2026 19:20
@samuelkarp samuelkarp dismissed their stale review March 6, 2026 19:21

I've amended the commit to fix the problem.

@dmcgowan dmcgowan moved this from Needs Update to Review In Progress in Pull Request Review Mar 6, 2026
@dmcgowan dmcgowan added the cherry-pick/2.2.x Change to be cherry picked to release/2.2 branch label Mar 6, 2026
@dmcgowan dmcgowan added this pull request to the merge queue Mar 6, 2026
Merged via the queue into containerd:main with commit 72663d6 Mar 6, 2026
54 checks passed
@github-project-automation github-project-automation Bot moved this from Review In Progress to Done in Pull Request Review Mar 6, 2026
@samuelkarp
Copy link
Copy Markdown
Member

/cherrypick release/2.2

@k8s-infra-cherrypick-robot
Copy link
Copy Markdown

@samuelkarp: new pull request created: #12987

Details

In response to this:

/cherrypick release/2.2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@akhilerm akhilerm deleted the fix-mirror-migration branch March 7, 2026 03:10
@chrishenzie chrishenzie added cherry-picked/2.2.x PR commits are cherry-picked into release/2.2 branch and removed cherry-pick/2.2.x Change to be cherry picked to release/2.2 branch labels Apr 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/cri Container Runtime Interface (CRI) cherry-picked/2.2.x PR commits are cherry-picked into release/2.2 branch size/S

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

cri plugin fails to load when using registry.mirrors with config version 2

6 participants