FROM --platform=$BUILDPLATFORM image ignores platform if image is a previous stage #3034

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

When running a multi-platform build with cross compilation, FROM --platform=$BUILDPLATFORM previous-stage ignores --platform=$BUILDPLATFORM if previous-stage didn't originate from a base image that used --platform=$BUILDPLATFORM. In other words, you can't build an earlier stage for both platforms and then use only the local platform version of it in a subsequent stage.

Expected behaviour

The documentation is silent on this, so I'm not sure whether above is expected behavior or not. Given this Dockerfile and entrypoint:

Dockerfile

# Runs on both platforms
FROM busybox:latest AS runtime-setup
RUN mkdir /r
RUN echo uname=$(uname -m) > /r/1

# Runs only on build platform
FROM busybox:latest AS build-setup
RUN mkdir /b
RUN echo uname=$(uname -m) > /b/1

# Runs on both platforms: --from=$BUILDPLATFORM is ignored
FROM --platform=$BUILDPLATFORM build-setup AS cross
ARG TARGETPLATFORM
RUN echo build=$(uname -m) target=$TARGETPLATFORM > /b/2

# Runs on both platforms; copies from the same platform
FROM runtime-setup AS runtime
COPY --from=cross /b /b
COPY --chmod=0755 entrypoint /entrypoint
ENTRYPOINT [ "/entrypoint" ]

entrypoint

#!/bin/sh
for i in /r/* /b/*; do
    echo $i
    cat $i
done

the statements in the cross stage are run on all platforms, not just $BUILDPLATFORM. I am guessing that it's because build-setup runs on both platforms.

For example, if you create an image with BUILDPLATFORM=linux/amd64 with the following commands:

docker build --platform linux/arm64,linux/amd64 -t m1 .
docker run --platform linux/arm64 --rm -ti m1

you get the following:

/r/1
uname=aarch64
/b/1
uname=aarch64
/b/2
build=aarch64 target=linux/arm64

The contents of /b/2 indicate that the "cross" step was run emulated on amd64. On the other hand, if you use this modified Dockerfile:

# Runs on both platforms
FROM busybox:latest AS runtime-setup
RUN mkdir /r
RUN echo uname=$(uname -m) > /r/1

# Runs only on build platform
FROM busybox:latest AS build-setup
RUN mkdir /b
RUN echo uname=$(uname -m) > /b/1

# Runs only on build platform because of qualified base image
FROM --platform=$BUILDPLATFORM build-setup:latest AS cross
ARG TARGETPLATFORM
RUN echo build=$(uname -m) target=$TARGETPLATFORM > /b/2

# Runs on both platforms; copies from the same platform
FROM runtime-setup AS runtime
COPY --from=cross /b /b
COPY --chmod=0755 entrypoint /entrypoint
ENTRYPOINT [ "/entrypoint" ]

(the only difference is replacing build-setup with build-setup:latest) and run these commands:

docker build --platform linux/arm64,linux/amd64 -t build-setup:latest --target build-setup .
docker build --platform linux/arm64,linux/amd64 . -t m2
docker run --platform linux/arm64 --rm -ti m2

You get this output:

/r/1
uname=aarch64
/b/1
uname=x86_64
/b/2
build=x86_64 target=linux/arm64

which indicates that the build platform version of the build-setup image was used.

This demonstrates that FROM --platform=$BUILDPLATFORM is honored when the image is a fully-qualified image (the usual for cross-platform builds), but not when it is a previous build stage that was not itself a build platform-only build.

I have reproduced this on Ubuntu 24.04 with the latest docker and /etc/docker/daemon.conf containing

{
  "features": {
    "containerd-snapshotter": true
  }
}

and also on macOS with colima configured to use containerd-snapshotter, though in that case, you get the opposite results since the "native" platform is arm64. On the mac, the first docker image run on amd64 shows

/r/1
uname=x86_64
/b/1
uname=x86_64
/b/2
build=x86_64 target=linux/amd64

and the second one shows

/r/1
uname=x86_64
/b/1
uname=aarch64
/b/2
build=aarch64 target=linux/amd64

Actual behaviour

I would have expected the first and second files to both behave like the second file, but I don't really know whether my expectations are incorrect. However, I would say there is one of two problems:

• If the above behavior is expected:
  • The build should warn (or even error) that it is ignoring the --from=$BUILDPLATFORM option when the image is a previous build stage
  • The documentation should be clear that the --from=<platform> option is not allowed when <image> is a previous build stage rather than a fully-qualified image.
• If the behavior is unexpected, then it's a bug -- but changing it would be potentially breaking, so I'm not sure what you should do.

Buildx version

github.com/docker/buildx v0.21.1 7c2359c AND github.com/docker/buildx v0.21.1 Homebrew

Docker info

Linux:

Client: Docker Engine - Community
 Version:    28.0.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.21.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.33.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 28.0.1
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.2.4-0-g6c52b3f
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-53-generic
 Operating System: Ubuntu 24.04.2 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 32
 Total Memory: 30.56GiB
 Name: soup
 ID: 483aedeb-f7b0-40cd-813d-92fefb8aa30d
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false


mac:

Client: Docker Engine - Community
 Version:    28.0.0
 Context:    colima
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.21.1
    Path:     /Users/ejb/.docker/cli-plugins/docker-buildx

Server:
 Containers: 35
  Running: 35
  Paused: 0
  Stopped: 0
 Images: 25
 Server Version: 27.4.0
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 88bf19b2105c8b17560993bee28a01ddc2f97182
 runc version: v1.2.2-0-g7cb3632
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-50-generic
 Operating System: Ubuntu 24.04.1 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 15.58GiB
 Name: colima
 ID: 3d4b77ba-75d4-43bd-b112-98d29b0380d3
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Builders list

Linux:

NAME/NODE     DRIVER/ENDPOINT                   STATUS     BUILDKIT   PLATFORMS
multi*        docker-container
 \_ multi0     \_ unix:///var/run/docker.sock   inactive
default       docker
 \_ default    \_ default                       running    v0.20.0    linux/amd64 (+4), linux/arm64, linux/arm (+2), linux/ppc64le, (6 more)

mac:

NAME/NODE    DRIVER/ENDPOINT    STATUS    BUILDKIT   PLATFORMS
multi*       docker-container
 \_ multi0    \_ colima         running   v0.19.0    linux/arm64
colima       docker
 \_ colima    \_ colima         running   v0.17.3    linux/arm64
default                         error

Configuration

# Run build with --platform linux/arm64,linux/amd64

# Runs on both platforms
FROM busybox:latest AS runtime-setup
RUN mkdir /r
RUN echo uname=$(uname -m) > /r/1

# Runs only on build platform
FROM busybox:latest AS build-setup
RUN mkdir /b
RUN echo uname=$(uname -m) > /b/1

# Runs on both platforms: --from=$BUILDPLATFORM is ignored
FROM --platform=$BUILDPLATFORM build-setup AS cross
ARG TARGETPLATFORM
RUN echo build=$(uname -m) target=$TARGETPLATFORM > /b/2

# Runs on both platforms; copies from the same platform
FROM runtime-setup AS runtime
COPY --from=cross /b /b
COPY --chmod=0755 entrypoint /entrypoint
ENTRYPOINT [ "/entrypoint" ]

See "Expected Behaviour" for detailed recipe for repoducing issue.

Build logs

Additional info

The documentation is silent on whether or not --from=<platform> works. As mentioned above, we should either

• document this behavior and, ideally, warn or give an error if it is seen, or
• make it work

Why am I doing this, you may ask? I'm creating an image that is used in two ways:

1. To efficiently build multi-platform output files using cross compilation
2. To run (with a source tree mounted into the container) builds and integration tests efficiently on multiple native platforms.

The functionality I'm seeing:

• Create a runtime chain of stages that run on both platforms
• Create a build image chain of stages that create a multi-platform build image. The build image should exist on multiple platforms, but creating the build image requires cross-compiling some third-party libraries
• Using the build image on the build platform to do additional steps

I was hoping for

• create runtime-setup stage; install base packages to be present on the runtime image
• create build-setup stage; install base packages and cross compiler tools to be present on the build image
• create cross-setup stage to run ONLY on build platform based on BUILDPLATFORM version of build-setup
• create build-image stage FROM build-setup stage (both platforms) and copy files created by the cross-setup stage
• create app-build stage FROM build-image on BUILDPLATFORM to natively and cross compile application
• create runtime stage FROM runtime-setup; copy for both platforms from the app-build-stage
• export runtime image (multiplatform)
• export build-image image (multiplatform)
• run build-image stage separately on multiple platforms, mounting in a source tree, and executing unit and integration tests

I can make it all work by explicitly building the build-image stage and exporting it as described, but I was hoping it could all be in one docker image.