Skip to content

daemon/RegistryHosts: Don't lose mirrors#46565

Merged
thaJeztah merged 2 commits intomoby:masterfrom
vvoland:c8d-mirrors-fix
Oct 13, 2023
Merged

daemon/RegistryHosts: Don't lose mirrors#46565
thaJeztah merged 2 commits intomoby:masterfrom
vvoland:c8d-mirrors-fix

Conversation

@vvoland
Copy link
Contributor

@vvoland vvoland commented Sep 29, 2023

docker.io is present in the IndexConfigs so the Mirrors property would get lost because a fresh RegistryConfig object was created.

Instead of creating a new object, reuse the existing one and just mutate its fields.

- What I did
Fixed mirrors not working with buildkit.

- How I did it
See commit.

- How to verify it
Try to use registry mirror.

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

@vvoland vvoland added area/distribution Image Distribution status/2-code-review kind/bugfix PR's that fix bugs labels Sep 29, 2023
@vvoland vvoland added this to the 25.0.0 milestone Sep 29, 2023
@vvoland
Copy link
Contributor Author

vvoland commented Sep 29, 2023

Note: Mirrors are still broken for buildkit and containerd integration on master (doesn't affect 24.0) until moby/buildkit#4293 is merged and gets vendored into moby.

We should probably block the 25.0 release before this is resolved.

@vvoland vvoland mentioned this pull request Sep 29, 2023
26 tasks
@vvoland
Copy link
Contributor Author

vvoland commented Sep 29, 2023

    build_test.go:1354: 
        	Error Trace:	/src/client/build_test.go:1354
        	            				/src/frontend/gateway/grpcclient/client.go:214
        	            				/src/client/build.go:59
        	            				/src/client/solve.go:283
        	            				/src/vendor/golang.org/x/sync/errgroup/errgroup.go:75
        	            				/usr/local/go/src/runtime/asm_amd64.s:1650
        	Error:      	Received unexpected error:
        	            	failed to load cache key: failed to do request: Head "https://localhost:37561/v2/library/busybox/manifests/latest?ns=docker.io": http: server gave HTTP response to HTTPS client
        	            	github.com/moby/buildkit/util/stack.Enable
        	            		/src/util/stack/stack.go:77
        	            	github.com/moby/buildkit/util/grpcerrors.FromGRPC
        	            		/src/util/grpcerrors/grpcerrors.go:198
        	            	github.com/moby/buildkit/util/grpcerrors.UnaryClientInterceptor
        	            		/src/util/grpcerrors/intercept.go:41
        	            	google.golang.org/grpc.(*ClientConn).Invoke
        	            		/src/vendor/google.golang.org/grpc/call.go:35
        	            	github.com/moby/buildkit/frontend/gateway/pb.(*lLBBridgeClient).StatFile
        	            		/src/frontend/gateway/pb/gateway.pb.go:2903
        	            	github.com/moby/buildkit/client.(*gatewayClientForBuild).StatFile
        	            		/src/client/build.go:110
        	            	github.com/moby/buildkit/frontend/gateway/grpcclient.(*reference).StatFile
        	            		/src/frontend/gateway/grpcclient/client.go:1131
        	            	github.com/moby/buildkit/client.testClientSlowCacheRootfsRef.func1
        	            		/src/client/build_test.go:1351
        	            	github.com/moby/buildkit/frontend/gateway/grpcclient.(*grpcClient).Run
        	            		/src/frontend/gateway/grpcclient/client.go:214
        	            	github.com/moby/buildkit/client.(*Client).Build.func2
        	            		/src/client/build.go:59
        	            	github.com/moby/buildkit/client.(*Client).solve.func3
        	            		/src/client/solve.go:283
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:75
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1650
        	Test:       	TestClientGatewayIntegration/TestClientSlowCacheRootfsRef/worker=dockerd

This will go away when buildkit PR is merged.

@thaJeztah
Copy link
Member

I guess for this one we need the moby/buildkit#4297 backport merged and vendored, or is that only for Buildkit standalone?

`docker.io` is present in the `IndexConfigs` so the `Mirrors` property
would get lost because a fresh `RegistryConfig` object was created.

Instead of creating a new object, reuse the existing one and just
mutate its fields.

Signed-off-by: Paweł Gronowski <[email protected]>
@vvoland
Copy link
Contributor Author

vvoland commented Oct 11, 2023

Revendored buildkit from the v0.12 release branch (not a tagged release yet).

Comment on lines +199 to 203
c := m[k]
if !v.Secure {
t := true
c.PlainHTTP = &t
c.Insecure = &t
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we have checks elsewhere, but does this mean we allow docker.io to be used with "insecure"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't because docker.io is always overriden with Secure: true entry:

moby/registry/config.go

Lines 236 to 241 in aacd100

indexConfigs[IndexName] = &registry.IndexInfo{
Name: IndexName,
Mirrors: config.Mirrors,
Secure: true,
Official: true,
}

but... I think it could be worked around if you explicitly use registry-1.docker.io? But it might get translated to docker.io before this is used, so perhaps it's not actually possible.

}
if _, ok := m[host]; !ok && daemon.registryService.IsInsecureRegistry(host) {
c := resolverconfig.RegistryConfig{}
if c, ok := m[host]; !ok && daemon.registryService.IsInsecureRegistry(host) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not something new in this PR, but IsInsecureRegistry also appears to be checking config.IndexConfigs. Are we doing the same thing twice in this function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, good question, I thought it was needed because it also does the isCIDRMatch check for insecure registries specified by CIDR (InsecureRegistryCIDRs).
But if I understand correctly, it actually doesn't handle them, because these are not a part of IndexConfigs anyway...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the overall combination of mirrors and insecure registries (and how they're both "separate" and "depending on each other") is ... messy

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's bring this one in; we can look at cleaning up the double check in a follow-up (not a new issue)

LGTM

@thaJeztah thaJeztah merged commit 80a9fc6 into moby:master Oct 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/distribution Image Distribution kind/bugfix PR's that fix bugs status/2-code-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants