Skip to content

Incorrect Layer SHA256 computation on large, self hosted containers #4908

@alexanderturner

Description

@alexanderturner

Description
I have a larger container which I have self-hosted in a Docker registry running registry:2. When I pull a particular container running graphite the deployment failed in k3s with an unexpected commit digest error. This only appears to happen on a private repo.

Steps to reproduce the issue:

  1. Deploy graphite.yaml below onto Kube running contained (k3s)
  2. kubectl describe pod graphite
apiVersion: v1
kind: Pod
metadata:
  name: graphite
  labels:
    app: graphite
    face: external
spec:
  containers:
  - name: graphite
    image: docker-public.fluid.aseit.com.au/graphite:1.1.7-6
    imagePullPolicy: Always
    ports:
      - containerPort: 80
      - containerPort: 2003
      - containerPort: 2004

Describe the results you received:
The pod always falls into ImageBackoff status

Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Normal   Scheduled  17s   default-scheduler  Successfully assigned default/graphite to alex
  Normal   Pulling    17s   kubelet            Pulling image "docker-public.fluid.aseit.com.au/graphite:1.1.7-6"
  Warning  Failed     0s    kubelet            Failed to pull image "docker-public.fluid.aseit.com.au/graphite:1.1.7-6": rpc error: code = FailedPrecondition desc = failed to pull and unpack image "docker-public.fluid.aseit.com.au/graphite:1.1.7-6": failed commit on ref "layer-sha256:f9a569415da5aad8ff56d04fdbc1c23a3ed2d4fd577e43f9d9e34e08c4b811da": unexpected commit digest sha256:0e258385e7eaba4e84dcfaf61c2c92b1d345c59645cbfc5d24ca7c90d023bd17, expected sha256:f9a569415da5aad8ff56d04fdbc1c23a3ed2d4fd577e43f9d9e34e08c4b811da: failed precondition
  Warning  Failed     0s    kubelet            Error: ErrImagePull

Describe the results you expected:

The image is pulled successfully.

Output of containerd --version:
Running as part of k3s

containerd-shim-runc-v2:
  Version:  1.4.3+unknown
  Revision:
  Go version: go1.15.5

Any other relevant information:
When pulling the image directly from the registry's /v2 endpoint I'm able to decompress it as well as compute the correct SHA256, as expected.

root@alex:/home/alex# wget https://docker-public.fluid.aseit.com.au/v2/graphite/blobs/sha256:f9a569415da5aad8ff56d04fdbc1c23a3ed2d4fd577e43f9d9e34e08c4b811da
--2021-01-06 05:21:27--  https://docker-public.fluid.aseit.com.au/v2/graphite/blobs/sha256:f9a569415da5aad8ff56d04fdbc1c23a3ed2d4fd577e43f9d9e34e08c4b811da
Resolving docker-public.fluid.aseit.com.au (docker-public.fluid.aseit.com.au)... 113.29.244.30
Connecting to docker-public.fluid.aseit.com.au (docker-public.fluid.aseit.com.au)|113.29.244.30|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 134427484 (128M) [application/octet-stream]
Saving to: ‘sha256:f9a569415da5aad8ff56d04fdbc1c23a3ed2d4fd577e43f9d9e34e08c4b811da’

sha256:f9a569415da5aad8ff56d04fdbc1c 100%[====================================================================>] 128.20M  10.3MB/s    in 12s

2021-01-06 05:21:39 (10.7 MB/s) - ‘sha256:f9a569415da5aad8ff56d04fdbc1c23a3ed2d4fd577e43f9d9e34e08c4b811da’ saved [134427484/134427484]

root@alex:/home/alex# sha256sum sha256\:f9a569415da5aad8ff56d04fdbc1c23a3ed2d4fd577e43f9d9e34e08c4b811da
f9a569415da5aad8ff56d04fdbc1c23a3ed2d4fd577e43f9d9e34e08c4b811da  sha256:f9a569415da5aad8ff56d04fdbc1c23a3ed2d4fd577e43f9d9e34e08c4b811da
root@alex:/home/alex#

This has been mentioned in the past here, here and most recently here - deleting ingest doesn't seem to resolve this, infact - the directory is empty.

root@alex:/home/alex# ls /var/lib/rancher/k3s/agent/containerd/io.containerd.content.v1.content/ingest/
root@alex:/home/alex#

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions