Skip to content

build: multi-node build support#37

Merged
tonistiigi merged 6 commits intomasterfrom
multi-driver
Apr 24, 2019
Merged

build: multi-node build support#37
tonistiigi merged 6 commits intomasterfrom
multi-driver

Conversation

@tonistiigi
Copy link
Member

@tonistiigi tonistiigi commented Apr 22, 2019

based on #35

Automatically split a build to chunks based on platform if needed, distribute between nodes and merge back to a manifest list when all nodes have completed.

@tiborvass
Copy link
Collaborator

tiborvass commented Apr 23, 2019 •
edited
Loading

We should add a TODO for cross repo. In manifest cli, it's implemented here https://github.com/docker/cli/blob/1fd2d66/cli/command/manifest/push.go#L120

Because that was missing I need to first push all the blobs to the same repo, which made me discover the following bug:

/work # docker run -d -p 5000:5000 --restart=always --name registry registry:2

/work # docker pull alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907
sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907: Pulling from library/alpine
bdf0201b3a05: Pull complete
Digest: sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907
Status: Downloaded newer image for alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907
docker.io/library/alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907

/work # docker tag alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 localhost:5000/alpine

/work # docker push localhost:5000/alpine
The push refers to repository [localhost:5000/alpine]
a464c54f93a9: Pushed
latest: digest: sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 size: 528

/work # docker pull alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7: Pulling from library/alpine
6f37394be673: Pull complete
Digest: sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
Status: Downloaded newer image for alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
docker.io/library/alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7

/work # docker tag alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7 localhost:5000/alpine

/work # docker push localhost:5000/alpine
The push refers to repository [localhost:5000/alpine]
e64a3852ea35: Pushed
latest: digest: sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7 
size: 528

/work # docker buildx imagetools inspect --raw localhost:5000/alpine
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 1512,
      "digest": "sha256:660030ffd296125c8af2f25a34ab2c12ac3d7656b31ac40ccf52eb14cd1c4bf0"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 2688778,
         "digest": "sha256:6f37394be673296a0fdc21b819c5df40431baf7d3af121bee451726dd1457493"
      }
   ]
}

/work # docker buildx imagetools create --dry-run -t localhost:5000/alpine localhost:5000/alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 localhost:5000/alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
{
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "schemaVersion": 2,
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907",
         "size": 528,
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7",
         "size": 528,
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}

/work # docker buildx imagetools create -t localhost:5000/alpine localhost:5000/alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 localhost:5000/alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
localhost:5000/alpine

/work # echo $?
0

/work # docker buildx imagetools inspect --raw localhost:5000/alpine
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 1512,
      "digest": "sha256:660030ffd296125c8af2f25a34ab2c12ac3d7656b31ac40ccf52eb14cd1c4bf0"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 2688778,
         "digest": "sha256:6f37394be673296a0fdc21b819c5df40431baf7d3af121bee451726dd1457493"
      }
   ]
}

I expected the last inspect to be the same as the create dry-run.

@tonistiigi
Copy link
Member Author

tonistiigi commented Apr 23, 2019

We should add a TODO for cross repo

It already errors with proper message https://github.com/tonistiigi/buildx/pull/37/files#diff-033177668af833b9d683c8f2593121deR79

/work # docker run -d -p 5000:5000 --restart=always --name registry registry:2
Unable to find image 'registry:2' locally
2: Pulling from library/registry
c87736221ed0: Pull complete
1cc8e0bb44df: Pull complete
54d33bcb37f5: Pull complete
e8afc091c171: Pull complete
b4541f6d3db6: Pull complete
Digest: sha256:3b00e5438ebd8835bcfa7bf5246445a6b57b9a50473e89c02ecc8e575be3ebb5
Status: Downloaded newer image for registry:2
77ffe38a44b53a33a3faa7f540a9b87d1641a14c8a9c2e00ed12ecd00e41bf8b
/work # docker pull alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907
sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907: Pulling from library/alpine
bdf0201b3a05: Pull complete
Digest: sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907
Status: Downloaded newer image for alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907
docker.io/library/alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907
/work # docker tag alpine@sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 lo
calhost:5000/alpine
/work # docker push localhost:5000/alpine
The push refers to repository [localhost:5000/alpine]
a464c54f93a9: Pushed
latest: digest: sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 size: 528
/work # docker pull alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7: Pulling from library/alpine
6f37394be673: Pull complete
Digest: sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
Status: Downloaded newer image for alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
docker.io/library/alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
/work # docker tag alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7 lo
calhost:5000/alpine
/work # docker push localhost:5000/alpine
The push refers to repository [localhost:5000/alpine]
e64a3852ea35: Pushed
latest: digest: sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7 size: 528
/work # docker buildx imagetools inspect --raw localhost:5000/alpine
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 1512,
      "digest": "sha256:660030ffd296125c8af2f25a34ab2c12ac3d7656b31ac40ccf52eb14cd1c4bf0"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 2688778,
         "digest": "sha256:6f37394be673296a0fdc21b819c5df40431baf7d3af121bee451726dd1457493"
      }
   ]
}
/work # docker buildx imagetools create --dry-run -t localhost:5000/alpine localhost:5000/alpine@sha
256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 localhost:5000/alpine@sha256:bc
6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f
failed to parse source "localhost:5000/alpine@sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f", valid sources are digests, refereces and descriptors: invalid checksum digest length
/work # docker buildx imagetools create --dry-run -t localhost:5000/alpine localhost:5000/alpine@sha
256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 localhost:5000/alpine@sha256:bc
6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
{
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "schemaVersion": 2,
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907",
         "size": 528,
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7",
         "size": 528,
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}
/work # docker buildx imagetools create -t localhost:5000/alpine localhost:5000/alpine@sha256:5c40b3
c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 localhost:5000/alpine@sha256:bc6e6ad08312
deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7
localhost:5000/alpine
/work # docker buildx imagetools inspect --raw localhost:5000/alpine
{
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "schemaVersion": 2,
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907",
         "size": 528,
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:bc6e6ad08312deb806ff4bf805c2e24f422859ff3f2082b68336e9b983fbc2f7",
         "size": 528,
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}

@tonistiigi tonistiigi mentioned this pull request Apr 23, 2019
Closed
@tiborvass
Copy link
Collaborator

tiborvass commented Apr 23, 2019

Indeed i was on the wrong commit 🤦‍♂️

@tonistiigi tonistiigi merged commit e7e57b6 into master Apr 24, 2019
@tiborvass tiborvass deleted the multi-driver branch April 25, 2019 03:25
@crazy-max crazy-max mentioned this pull request Jul 23, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tiborvass tiborvass tiborvass approved these changes

@AkihiroSuda AkihiroSuda AkihiroSuda approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tonistiigi @tiborvass @AkihiroSuda