Skip to content

add support for building from cached images#3797

Merged
deitch merged 1 commit intolinuxkit:masterfrom
deitch:use-images-from-cache
Jul 20, 2022
Merged

add support for building from cached images#3797
deitch merged 1 commit intolinuxkit:masterfrom
deitch:use-images-from-cache

Conversation

@deitch
Copy link
Copy Markdown
Collaborator

@deitch deitch commented Jul 3, 2022

Signed-off-by: Avi Deitcher [email protected]

- What I did

Enable linuxkit to use images from the cache. This makes the behaviour exactly like docker, and resolves a very longstanding issue.

If you do linuxkit pkg build ./pkg/foo which outputs, say, linuxkit/foo:abcd (based on build.yaml), and you want to use that image in your next build without first pushing it to the registry, it wouldn't work. E.g.:

FROM linuxkit/foo:abcd

This adds support for reading it from the local cache. It also gives you the ability to override that and ignore the local cache with --ignore-cached, so the behaviour is:

  • lkt pkg build dir/ - build based on dir/, and for all FROM images in the dockerfile, first look in the linuxkit cache, and only after at the remote registry
  • lkt pkg build --ignore-cached dir/ - build based on dir/, but do not try to load any images from the local linuxkit cache.

- How I did it

Leverage the most recent #3791 to use recent improvements in buildkit to read from OCI layout caches.

- How to verify it

CI. I added a test for it.

- Description for the changelog

Ability to use images in linuxkit cache when building packages

@deitch deitch force-pushed the use-images-from-cache branch from 66b73d9 to 9e9d4c6 Compare July 3, 2022 09:14
@deitch
Copy link
Copy Markdown
Collaborator Author

deitch commented Jul 3, 2022

This is not quite going to be ready yet. We need to run a version of buildkit container that supports this. Unfortunately, there only is semver (most recent is v0.10.3, which is 2 months old and predates the support), master and nightly, which change constantly.

@deitch deitch force-pushed the use-images-from-cache branch from 9e9d4c6 to 5b46ed2 Compare July 3, 2022 12:05
@deitch
Copy link
Copy Markdown
Collaborator Author

deitch commented Jul 3, 2022

tests are correct now, but it will fail, because the buildkit image being run is v0.10.3. I don't want to tie this to unstable master; open question now with the buildkit folks; as soon as it is resolved, will complete this.

@deitch deitch force-pushed the use-images-from-cache branch from 5b46ed2 to 6d13af8 Compare July 6, 2022 09:28
@deitch deitch force-pushed the use-images-from-cache branch 4 times, most recently from 59e6855 to f3fa673 Compare July 14, 2022 08:50
@deitch deitch force-pushed the use-images-from-cache branch from f3fa673 to a90ff54 Compare July 14, 2022 09:54
@giggsoff giggsoff mentioned this pull request Jul 15, 2022
Closed
@giggsoff
Copy link
Copy Markdown
Contributor

giggsoff commented Jul 19, 2022

It is quite strange, but I cannot build for another platform (I am on amd64) when starting from empty ~/.linuxkit directory:

giggsoff@giggsoff-PC:~/linuxkit$ ./bin/linuxkit version
linuxkit version v0.8+
commit: a90ff542cd11a3eef0b3eb9657ebbac3c6e43531

giggsoff@giggsoff-PC:~/linuxkit$ rm -rf ~/.linuxkit/

giggsoff@giggsoff-PC:~/linuxkit$ ./bin/linuxkit pkg build --force --platforms=linux/arm64 ./test/cases/000_build/040_chained_builds/build1
Building "linuxkit/chained-build-image1-test-does-not-exist-anywhere-else-123456789:ad06eb29c48f0daab0cc2836a1351a053948100c"
building docker.io/linuxkit/chained-build-image1-test-does-not-exist-anywhere-else-123456789:ad06eb29c48f0daab0cc2836a1351a053948100c
Building for arch arm64 as linuxkit/chained-build-image1-test-does-not-exist-anywhere-else-123456789:ad06eb29c48f0daab0cc2836a1351a053948100c-arm64
using existing container linuxkit-builder
building for platform linux/arm64
[+] Building 7.3s (7/7)                                                                                                                                                                                                                           
 => CACHED [internal] load remote build context                                                                                                                                                                                              1.1s
 => CACHED copy /context /                                                                                                                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/alpine:3.15                                                                                                                                                                               4.7s
 => [1/2] FROM docker.io/library/alpine:3.15@sha256:4edbd2beb5f78b1014028f4fbb99f3237d9561100b6881aabbf5acce2c4f9454                                                                                                                         0.6s
 => => resolve docker.io/library/alpine:3.15@sha256:4edbd2beb5f78b1014028f4fbb99f3237d9561100b6881aabbf5acce2c4f9454                                                                                                                         0.6s
 => CACHED copy /context /                                                                                                                                                                                                                   0.0s
 => CACHED [2/2] COPY foo /foo                                                                                                                                                                                                               0.0s
 => exporting to oci image format                                                                                                                                                                                                            0.8s
 => => exporting layers                                                                                                                                                                                                                      0.0s
 => => exporting manifest sha256:26d9e79e8ba054f576f29f553bbf47d842ed99877c01f9b03e9de37ca5897c9b                                                                                                                                            0.3s
 => => exporting config sha256:f5917beabe31cef552d2e8b879f917fb1969ed64fedb503e4d92cb759dfa90b5                                                                                                                                              0.3s
 => => sending tarball                                                                                                                                                                                                                       0.0s
Build complete, not pushing, all done.

giggsoff@giggsoff-PC:~/linuxkit$ ./bin/linuxkit pkg build --force --platforms=linux/arm64 ./test/cases/000_build/040_chained_builds/build2
Building "linuxkit/chained-build-test-second:cbc75900bf7df82ca3122cd46f21fe21d2efc7cd"
building docker.io/linuxkit/chained-build-test-second:cbc75900bf7df82ca3122cd46f21fe21d2efc7cd
Building for arch arm64 as linuxkit/chained-build-test-second:cbc75900bf7df82ca3122cd46f21fe21d2efc7cd-arm64
using existing container linuxkit-builder
building for platform linux/arm64
[+] Building 12.4s (5/8)                                                                                                                                                                                                                          
 => CACHED [internal] load remote build context                                                                                                                                                                                              1.1s
 => CACHED copy /context /                                                                                                                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/alpine:3.15                                                                                                                                                                               1.3s
 => [context linuxkit/chained-build-image1-test-does-not-exist-anywhere-else-123456789:ad06eb29c48f0daab0cc2836a1351a053948100c] load metadata for linuxkit-cache@sha256:bb20a8383e1b0222ff87aa21cadf9f8b0a108d51ba79c6eee213a464f5b3f16f   10.6s
 => ERROR [context linuxkit/chained-build-image1-test-does-not-exist-anywhere-else-123456789:ad06eb29c48f0daab0cc2836a1351a053948100c] OCI load from client                                                                                  0.3s
 => => resolve oci-layout/dummy@sha256:bb20a8383e1b0222ff87aa21cadf9f8b0a108d51ba79c6eee213a464f5b3f16f                                                                                                                                      0.2s
 => [stage-1 1/3] FROM docker.io/library/alpine:3.15@sha256:4edbd2beb5f78b1014028f4fbb99f3237d9561100b6881aabbf5acce2c4f9454                                                                                                                 0.7s
 => => resolve docker.io/library/alpine:3.15@sha256:4edbd2beb5f78b1014028f4fbb99f3237d9561100b6881aabbf5acce2c4f9454                                                                                                                         0.5s
Error building "linuxkit/chained-build-test-second:cbc75900bf7df82ca3122cd46f21fe21d2efc7cd": error building for arch arm64: failed to solve: failed to load cache key: no match for platform in manifest: not found

@deitch
Copy link
Copy Markdown
Collaborator Author

deitch commented Jul 20, 2022

Really strange @giggsoff ; I cannot replicate it. I tried both with and without --platforms=linux/arm64, i.e. restricting it, or all supported platforms.

@deitch
Copy link
Copy Markdown
Collaborator Author

deitch commented Jul 20, 2022

We are going to merge this in soon. Once it is, if you can replicate it, we will open a new issue to focus on it.

@giggsoff giggsoff mentioned this pull request Jul 20, 2022
Merged
@deitch
Copy link
Copy Markdown
Collaborator Author

deitch commented Jul 20, 2022

been open for almost 3 weeks, CI green for a week, and @justincormack wrote:

I am fine merging things people don't object to that pass CI

If anyone has issues with this, take it up with me on Slack.

@deitch deitch merged commit 403e8f9 into linuxkit:master Jul 20, 2022
@deitch deitch deleted the use-images-from-cache branch July 20, 2022 13:29
@giggsoff giggsoff mentioned this pull request Jul 22, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@deitch @giggsoff