Dockerfile: use TARGETPLATFORM by crazy-max · Pull Request #44513 · moby/moby

crazy-max · 2022-11-23T14:04:40Z

follow-up #43529

Prepare next follow-ups to properly handle cross compilation using TARGETPLATFORM global arg.

This will optimize build for each components as it doesn't depend anymore on dev-base stage with hardcoded linux/amd64 platform which installs cross packages for all platforms for cross compilation:

moby/Dockerfile

Lines 133 to 149 in d6d0e4c

    
           FROM --platform=linux/amd64 runtime-dev-cross-false AS runtime-dev-cross-true 
        
           ARG DEBIAN_FRONTEND 
        
           # These crossbuild packages rely on gcc-<arch>, but this doesn't want to install 
        
           # on non-amd64 systems, so other architectures cannot crossbuild amd64. 
        
           RUN --mount=type=cache,sharing=locked,id=moby-cross-true-aptlib,target=/var/lib/apt \ 
        
               --mount=type=cache,sharing=locked,id=moby-cross-true-aptcache,target=/var/cache/apt \ 
        
                   apt-get update && apt-get install -y --no-install-recommends \ 
        
                       libapparmor-dev:arm64 \ 
        
                       libapparmor-dev:armel \ 
        
                       libapparmor-dev:armhf \ 
        
                       libapparmor-dev:ppc64el \ 
        
                       libapparmor-dev:s390x \ 
        
                       libseccomp-dev:arm64 \ 
        
                       libseccomp-dev:armel \ 
        
                       libseccomp-dev:armhf \ 
        
                       libseccomp-dev:ppc64el \ 
        
                       libseccomp-dev:s390x

In follow-up remove CROSS and DOCKER_CROSSPLATFORMS arg and cross stages in Dockerfile. It also needs changes in hack scripts.

cc @rumpl @vvoland @thaJeztah

Dockerfile

thaJeztah

Ah, crap. Had a review that I didn't commit; comments may be outdated by now (haven't checked)

thaJeztah · 2022-11-24T16:18:57Z

Dockerfile

+# runc
+FROM base AS runc-src
+WORKDIR /usr/src/runc
+RUN git init . && git remote add origin "https://github.com/opencontainers/runc.git"


Should we start dogfooding dockerfile (labs): implement ADD <git ref> buildkit#2799 ?

Good idea, maybe for a follow-up and wait for final release of Dockerfile frontend 1.5?

Dockerfile

thaJeztah · 2022-11-26T14:31:40Z

Dockerfile

-                   go build -buildmode=pie -o /build/registry-v2-schema1 github.com/docker/distribution/cmd/registry; \
-                ;; \
-           esac
+    --mount=type=tmpfs,target=/go/src <<EOT


Should this use <<-EOT, if the intent is to remove the first indentation? (we may have to change some spaces to tabs for that)

Dockerfile

thaJeztah · 2022-11-29T10:57:15Z

Dockerfile

-  export CGO_ENABLED=0
+  export CGO_ENABLED=$([ "$DOCKER_STATIC" = "1" ] && echo "0" || echo "1")


Was this still needed? (if so; should we fix the containerd makefile to set it automatically?)

I see there's also a SHIM_CGO_ENABLED (but not sure if there's anything we need to do with that one); https://github.com/containerd/containerd/blob/e0be97ccee444c06dc1268df5a99ed80c2cfea76/Makefile#L36

Yes needed otherwise ctr is not static and containerd build warns for native builds:

$ docker buildx build --target containerd . ... #22 [containerd-build 3/3] RUN --mount=from=containerd-src,src=/usr/src/containerd,rw --mount=type=cache,target=/root/.cache <<EOT (set -e...) #22 0.862 + bin/ctr #22 1.921 + bin/containerd #22 3.631 # github.com/containerd/containerd/cmd/containerd #22 3.631 /usr/bin/ld: /tmp/go-link-508382094/000019.o: in function `New': #22 3.631 /go/src/github.com/containerd/containerd/vendor/github.com/miekg/pkcs11/pkcs11.go:77: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking #22 3.712 + bin/containerd-stress #22 4.438 + bin/containerd-shim #22 4.885 + bin/containerd-shim-runc-v1 #22 5.423 + bin/containerd-shim-runc-v2 #22 5.767 + binaries #22 DONE 5.9s

Looks like something we should fix in containerd; they added the STATIC=1 make var; I think that one should ideally take care of that automatically.

Dockerfile

thaJeztah · 2022-12-15T13:49:27Z

Dockerfile

+  xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /build/rootlesskit
+  xx-go build -o /build/rootlesskit-docker-proxy -ldflags="$([ "$DOCKER_STATIC" != "1" ] && echo "-linkmode=external")" ./cmd/rootlesskit-docker-proxy
+  xx-verify $([ "$DOCKER_STATIC" = "1" ] && echo "--static") /build/rootlesskit-docker-proxy


(not for this PR); perhaps we should have a xx-verify utility that looks for an environment variable, so that we can use XX_STATIC=1 and run xx-verify unconditionally (would save boiler plating in quite some projects now I think) 🤔

Dockerfile

Signed-off-by: CrazyMax <[email protected]>

thaJeztah · 2022-12-15T17:47:00Z

some failures in docker-py; could be some race condition, but posting, as we recently merged some fixes for docker ps in #44629 (in case it's related)

____________________________ ContainerTest.test_top ____________________________
tests/integration/models_containers_test.py:424: in test_top
    assert len(top['Processes']) == 1
E   AssertionError: assert 3 == 1
E    +  where 3 = len([['root', '32619', '32599', '0', '14:20', '?', ...], ['root', '32619', '32599', '0', '14:20', '?', ...], ['root', '32619', '32599', '0', '14:20', '?', ...]])
------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------
=========================== short test summary info ============================
XFAIL tests/integration/api_container_test.py::CreateContainerTest::test_create_with_cpu_rt_options
  CONFIG_RT_GROUP_SCHED isn't enabled
XFAIL tests/integration/api_container_test.py::CreateContainerTest::test_create_with_storage_opt
  Not supported on most drivers
XFAIL tests/integration/api_container_test.py::ContainerTopTest::test_top
  Output of docker top depends on host distro, and is not formalized.
XFAIL tests/integration/api_container_test.py::ContainerTopTest::test_top_with_psargs
  Output of docker top depends on host distro, and is not formalized.
XFAIL tests/integration/api_swarm_test.py::SwarmTest::test_init_swarm_data_path_addr
  Can fail if eth0 has multiple IP addresses
XFAIL tests/integration/api_swarm_test.py::SwarmTest::test_init_swarm_with_log_driver
  This doesn't seem to be taken into account by the engine
SKIPPED [1] tests/integration/api_image_test.py:291: Doesn't work inside a container - FIXME
SKIPPED [1] /src/tests/integration/api_swarm_test.py:31: Test stalls the engine on 1.12.0
= 1 failed, 381 passed, 2 skipped, 3 deselected, 6 xfailed, 2 xpassed in 257.39 seconds =

thaJeztah

LGTM, thanks!

corhere

LGTM

thaJeztah · 2022-12-15T22:00:10Z

Okay; let's get this one in, thanks! I'll push my cherry-pick PR

crazy-max commented Nov 23, 2022

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

crazy-max force-pushed the dockerfile-targetplatform branch 5 times, most recently from a917f3f to 170e1f6 Compare November 24, 2022 11:02

crazy-max commented Nov 24, 2022

View reviewed changes

Dockerfile Show resolved Hide resolved

crazy-max force-pushed the dockerfile-targetplatform branch from 170e1f6 to 96c1201 Compare November 24, 2022 12:33

crazy-max marked this pull request as ready for review November 24, 2022 12:39

crazy-max requested a review from thaJeztah November 24, 2022 12:39

thaJeztah added status/2-code-review process/cherry-pick area/packaging labels Nov 24, 2022

thaJeztah added this to the v-next milestone Nov 24, 2022

crazy-max force-pushed the dockerfile-targetplatform branch 6 times, most recently from 80a5303 to 02276cd Compare November 26, 2022 13:29

thaJeztah reviewed Nov 26, 2022

View reviewed changes

crazy-max force-pushed the dockerfile-targetplatform branch 2 times, most recently from efc3683 to 0970234 Compare November 27, 2022 13:34

thaJeztah reviewed Nov 29, 2022

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

thaJeztah reviewed Nov 29, 2022

View reviewed changes

crazy-max force-pushed the dockerfile-targetplatform branch from 0970234 to c60ee82 Compare November 29, 2022 11:31

crazy-max mentioned this pull request Nov 29, 2022

Dockerfile: use TARGETPLATFORM to build Docker #44546

Merged

thaJeztah mentioned this pull request Dec 2, 2022

Use ARG in stage name moby/buildkit#2618

Open

thaJeztah reviewed Dec 3, 2022

View reviewed changes

Dockerfile Show resolved Hide resolved

Dockerfile Outdated Show resolved Hide resolved

Dockerfile Outdated Show resolved Hide resolved

thaJeztah reviewed Dec 15, 2022

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

crazy-max force-pushed the dockerfile-targetplatform branch 2 times, most recently from 2a81651 to 7298765 Compare December 15, 2022 13:59

crazy-max added 12 commits December 15, 2022 15:02

Dockerfile: add cross compilation helper

6cba7d2

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: add dummy stage

69c9dfd

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: DOCKER_STATIC arg

b19c116

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: containerd cross build with TARGETPLATFORM

81d704d

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: runc cross build with TARGETPLATFORM

5e19d7e

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: tini cross build with TARGETPLATFORM

f030789

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: rootlesskit cross build with TARGETPLATFORM

420591c

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: dockercli cross build with TARGETPLATFORM

a4bc292

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: registry cross build with TARGETPLATFORM

8d9a357

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: containerutility cross build with TARGETPLATFORM

879b023

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: swagger cross build with TARGETPLATFORM

ed95e1a

Signed-off-by: CrazyMax <[email protected]>

Dockerfile: delve cross build with TARGETPLATFORM

19f3448

Signed-off-by: CrazyMax <[email protected]>

crazy-max force-pushed the dockerfile-targetplatform branch from 7298765 to 19f3448 Compare December 15, 2022 14:02

crazy-max mentioned this pull request Dec 15, 2022

Check env var for xx-verify utility tonistiigi/xx#98

Closed

thaJeztah approved these changes Dec 15, 2022

View reviewed changes

thaJeztah mentioned this pull request Dec 15, 2022

update containerd binary to v1.7.0-beta.1 #44653

Merged

corhere approved these changes Dec 15, 2022

View reviewed changes

thaJeztah merged commit 5f1938f into moby:master Dec 15, 2022

thaJeztah mentioned this pull request Dec 15, 2022

[23.0 backport] Dockerfile: use TARGETPLATFORM #44659

Merged

thaJeztah added process/cherry-picked and removed process/cherry-pick labels Dec 15, 2022

crazy-max deleted the dockerfile-targetplatform branch December 15, 2022 22:23

crazy-max mentioned this pull request Jul 9, 2023

xx-verify: XX_VERIFY_STATIC env tonistiigi/xx#109

Merged

thaJeztah mentioned this pull request Dec 11, 2024

Dockerfile: remove libapparmor-dev dependency #49066

Merged

	FROM --platform=linux/amd64 runtime-dev-cross-false AS runtime-dev-cross-true
	ARG DEBIAN_FRONTEND
	# These crossbuild packages rely on gcc-<arch>, but this doesn't want to install
	# on non-amd64 systems, so other architectures cannot crossbuild amd64.
	RUN --mount=type=cache,sharing=locked,id=moby-cross-true-aptlib,target=/var/lib/apt \
	--mount=type=cache,sharing=locked,id=moby-cross-true-aptcache,target=/var/cache/apt \
	apt-get update && apt-get install -y --no-install-recommends \
	libapparmor-dev:arm64 \
	libapparmor-dev:armel \
	libapparmor-dev:armhf \
	libapparmor-dev:ppc64el \
	libapparmor-dev:s390x \
	libseccomp-dev:arm64 \
	libseccomp-dev:armel \
	libseccomp-dev:armhf \
	libseccomp-dev:ppc64el \
	libseccomp-dev:s390x

		export CGO_ENABLED=0
		export CGO_ENABLED=$([ "$DOCKER_STATIC" = "1" ] && echo "0" \|\| echo "1")

Conversation

crazy-max commented Nov 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

thaJeztah commented Dec 15, 2022

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

corhere left a comment

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Dec 15, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

crazy-max commented Nov 23, 2022 •

edited

Loading