Description
Found while testing buildpacks with the new storage backend here. We run an in memory registry with generated credentials for our acceptance test suite. When using ImagePush, the provided RegistryAuth is not provided to the target registry and the caller fails with a 401 Unauthorized.
Reproduce
Run registry with simple auth
docker run -it -p 5005:5000 -e PORT=5000 -e REGISTRY_AUTH=htpasswd -e REGISTRY_AUTH_HTPASSWD_REALM="Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd -v $(pwd):/auth registry:2
Run docker push via dockerCliVal.ImagePush
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/registry"
dockercli "github.com/docker/docker/client"
"github.com/docker/docker/pkg/jsonmessage"
"github.com/pkg/errors"
)
func main() {
dockerClient, err := dockercli.NewClientWithOpts(dockercli.FromEnv, dockercli.WithVersion("1.38"))
if err != nil {
panic(err)
}
basicAuth := registry.AuthConfig{
Username: "admin",
Password: "admin",
}
encodedAuth, err := registry.EncodeAuthConfig(basicAuth)
if err != nil {
panic(err)
}
result, err := dockerClient.ImagePush(context.Background(), "127.0.0.1:5005/test-alpine", types.ImagePushOptions{
RegistryAuth: encodedAuth,
Platform: "linux/amd64",
})
if err != nil {
panic(err)
}
defer result.Close()
err = checkResponseError(result)
if err != nil {
panic(err)
}
fmt.Println("Done")
}
func checkResponseError(r io.Reader) error {
responseBytes, err := io.ReadAll(r)
if err != nil {
return err
}
responseBuf := bytes.NewBuffer(responseBytes)
decoder := json.NewDecoder(responseBuf)
for {
var jsonMessage jsonmessage.JSONMessage
err := decoder.Decode(&jsonMessage)
if err != nil {
return fmt.Errorf("parsing response: %w\n%s", err, responseBuf.String())
}
if jsonMessage.Error != nil {
return errors.Wrap(jsonMessage.Error, "embedded daemon response")
}
if !decoder.More() {
break
}
}
return nil
}
➜ containerdstoragerissue go run main.go
panic: embedded daemon response: unexpected status from HEAD request to http://127.0.0.1:5005/v2/test-alpine/blobs/sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1: 401 Unauthorized
goroutine 1 [running]:
main.main()
/Users/jesse.brown/dev/scratch/containerdstoragerissue/main.go:48 +0x285
exit status 2
Expected behavior
I expect the push to succeed and it does on the overlayfs storage driver. A docker push does succeed after a docker login
docker version
Client:
Cloud integration: v1.0.35+desktop.5
Version: 24.0.6
API version: 1.43
Go version: go1.20.7
Git commit: ed223bc
Built: Mon Sep 4 12:28:49 2023
OS/Arch: darwin/amd64
Context: default
Server: Docker Desktop 4.24.2 (124339)
Engine:
Version: 24.0.6
API version: 1.43 (minimum version 1.12)
Go version: go1.20.7
Git commit: 1a79695
Built: Mon Sep 4 12:32:16 2023
OS/Arch: linux/amd64
Experimental: true
containerd:
Version: 1.6.22
GitCommit: 8165feabfdfe38c65b599c4993d227328c231fca
runc:
Version: 1.1.8
GitCommit: v1.1.8-0-g82f18fe
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client:
Version: 24.0.6
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2-desktop.5
Path: /Users/jesse.brown/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.22.0-desktop.2
Path: /Users/jesse.brown/.docker/cli-plugins/docker-compose
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.0
Path: /Users/jesse.brown/.docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.20
Path: /Users/jesse.brown/.docker/cli-plugins/docker-extension
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v0.1.0-beta.8
Path: /Users/jesse.brown/.docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /Users/jesse.brown/.docker/cli-plugins/docker-sbom
scan: Docker Scan (Docker Inc.)
Version: v0.26.0
Path: /Users/jesse.brown/.docker/cli-plugins/docker-scan
scout: Docker Scout (Docker Inc.)
Version: v1.0.7
Path: /Users/jesse.brown/.docker/cli-plugins/docker-scout
Server:
Containers: 21
Running: 0
Paused: 0
Stopped: 21
Images: 42
Server Version: dev
Storage Driver: stargz
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
runc version: v1.1.8-0-g82f18fe
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.4.16-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 15.63GiB
Name: docker-desktop
ID: 443b6e33-3385-4db7-8928-e2ff4057a16f
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Experimental: true
Insecure Registries:
192.168.185.110:5000
hubproxy.docker.internal:5555
192.168.185.110/32
192.168.185.112/32
127.0.0.0/8
Live Restore Enabled: false
Additional Info
No response
Description
Found while testing buildpacks with the new storage backend here. We run an in memory registry with generated credentials for our acceptance test suite. When using
ImagePush, the providedRegistryAuthis not provided to the target registry and the caller fails with a401 Unauthorized.Reproduce
Run registry with simple auth
Run
docker pushviadockerCliVal.ImagePushExpected behavior
I expect the push to succeed and it does on the overlayfs storage driver. A
docker pushdoes succeed after adocker logindocker version
Client: Cloud integration: v1.0.35+desktop.5 Version: 24.0.6 API version: 1.43 Go version: go1.20.7 Git commit: ed223bc Built: Mon Sep 4 12:28:49 2023 OS/Arch: darwin/amd64 Context: default Server: Docker Desktop 4.24.2 (124339) Engine: Version: 24.0.6 API version: 1.43 (minimum version 1.12) Go version: go1.20.7 Git commit: 1a79695 Built: Mon Sep 4 12:32:16 2023 OS/Arch: linux/amd64 Experimental: true containerd: Version: 1.6.22 GitCommit: 8165feabfdfe38c65b599c4993d227328c231fca runc: Version: 1.1.8 GitCommit: v1.1.8-0-g82f18fe docker-init: Version: 0.19.0 GitCommit: de40ad0docker info
Additional Info
No response