Better support building multi-arch Dockerfile via platform ARG mapping to chisel cut --arch

TL;DR: Have chisel cut --arch map/normalize BuildKit supplied platform ARGs to the appropriate --arch value, making cross-platform builds much simpler with docker build --platform (which is commonly used on GH CI with docker/build-push-action):

# Creating a container driver for Buildx to build with:
docker buildx create --name=container --driver=docker-container --bootstrap --use

# Now we can build multiple platforms concurrently:
docker buildx build \
  --builder=container \
  --platform=linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6,linux/386,linux/ppc64le \
  --tag localhost/testssl \
  .

Problem and workarounds

Using chisel to get packages for a different arch is fairly simple with chisel cut --arch (provided you know what the supported architecture values look like, otherwise for the unfamiliar there's ab it more legwork to search that up).

However, I recently looked into building a multi-arch image with Chisel beyond the common amd64/arm64 archs, and encountered a few points of friction.

While chisel cut --arch maps to the same archs that Ubuntu repos use:

The GH Releases omit several of those archs, so you cannot pull from there for a convenient executable when building non-native arch images by using TARGETARCH with the default docker build behaviour emulating that target arch.

# NOTE: This will be redundant once an official chisel image is released:
FROM alpine AS chisel
ARG CHISEL_VERSION=1.1.0
ARG TARGETARCH
ARG CHISEL_RELEASE="https://github.com/canonical/chisel/releases/download/v${CHISEL_VERSION}/chisel_v${CHISEL_VERSION}_linux_${TARGETARCH}.tar.gz"
RUN wget -qO - "${CHISEL_RELEASE}" | tar -xz --no-same-owner -C /usr/local/bin chisel

While this is typically slower, it does infer the platform arch emulated for chisel cut, removing any need for an explicit --arch. That's useful because of the next point..

chisel cut --arch does not map TARGETARCH (or related ARG from BuildKit) to the repo architectures expected / supported.

TARGETARCH can be arm, which matches the GH release arch. But that is presumably armhf (arm/v7), since unlike Debian, Ubuntu only has support for that arch. The legacy armel (arm/v6) would still be arm for TARGETARCH, you need to pair it with TARGETVARIANT to get the v7 vs v6 distinction. Regardless, these ARG values do not match the apt repo arch, so some logic is now required to map between the two.

Similar story for ppc64le => ppc64el and 386 => i386.

So we end up with something like this:

Use --platform=${BUILDPLATFORM} to only use a base image native to your build host for all targets, and likewise only pull a compatible binary for that (BASEARCH instead of TARGETARCH) which won't be required once there is an official chisel image.

# NOTE: This will be redundant once an official chisel image is released:
FROM --platform=${BUILDPLATFORM} alpine AS chisel
ARG CHISEL_VERSION=1.1.0
ARG BUILDARCH
ARG CHISEL_RELEASE="https://github.com/canonical/chisel/releases/download/v${CHISEL_VERSION}/chisel_v${CHISEL_VERSION}_linux_${BUILDARCH}.tar.gz"
RUN wget -qO - "${CHISEL_RELEASE}" | tar -xz --no-same-owner -C /usr/local/bin chisel

Map platforms to package arch:

# Workaround - Performance improvement (avoid emulation for non-native image builds)
# Map TARGETPLATFORM from BuildKit to equivalent input for `chisel cut --arch`:
FROM --platform=${BUILDPLATFORM} alpine AS chisel-arch-target
ARG TARGETPLATFORM
RUN <<HEREDOC
  case "${TARGETPLATFORM}" in
    ( 'linux/amd64' )
      echo 'amd64' > /target-arch.txt
      ;;
    ( 'linux/arm64' )
      echo 'arm64' > /target-arch.txt
      ;;
    ( 'linux/arm/v7' )
      echo 'armhf' > /target-arch.txt
      ;;
    ( 'linux/arm/v6' )
      # NOTE: This presumably isn't reliable and should be `armel` (if Ubuntu supported it):
      echo 'armhf' > /target-arch.txt
      ;;
    ( 'linux/386' )
      echo 'i386' > /target-arch.txt
      ;;
    ( 'linux/ppc64le' )
      echo 'ppc64el' > /target-arch.txt
      ;;
    ( * )
      echo "ERROR: Unsupported target platform: '${TARGETPLATFORM}'"
      return 1
      ;;
  esac
HEREDOC

or alternatively:

# The suffix matches TARGETARCH + TARGETVARIANT
# TARGETPLATFORM is not valid in the stage name due to presence of `/`
FROM chisel AS chisel-amd64
ENV CHISEL_ARCH=amd64
FROM chisel AS chisel-arm64
ENV CHISEL_ARCH=arm64
FROM chisel AS chisel-armv7
ENV CHISEL_ARCH=armhf
FROM chisel AS chisel-armv6
ENV CHISEL_ARCH=armel
FROM chisel AS chisel-386
ENV CHISEL_ARCH=i386
FROM chisel AS chisel-ppc64le
ENV CHISEL_ARCH=ppc64el

Have chisel cut --arch access that arch value from /target-arch.txt:

# Prepare root-fs with required packages:
FROM chisel AS root-fs
COPY --link --from=chisel-arch-target /target-arch.txt /target-arch.txt
WORKDIR /root-fs
RUN --mount=type=cache,target=/.cache/chisel,id="chisel-cache" \
    chisel cut --arch "$(cat /target-arch.txt)" --release ubuntu-24.04 --root /root-fs \
        base-files_base bash_bins

or with alternative pulling in the appropriate stage for the correct CHISEL_ARCH ENV:

# Prepare root-fs with required packages:
FROM chisel-${TARGETARCH}${TARGETVARIANT} AS root-fs
WORKDIR /root-fs
RUN --mount=type=cache,target=/.cache/chisel,id="chisel-cache" \
    chisel cut --arch "${CHISEL_ARCH?arch-required}" --release ubuntu-24.04 --root /root-fs \
        base-files_base bash_bins

You can now build for i386 (and for armhf & ppc64el more performantly), but it's awkward (I don't personally need this, but the referenced project was reluctant to drop supporting it).

This is something that could be improved on, so that examples like I showed here would work as intended without that mapping workaround demonstrated above.

chisel cut --arch should ideally recognize TARGETPLATFORM and TARGETARCH to normalize/map to the equivalent --arch value supported, or fail early when that is not a valid/supported arch (if possible). Should chisel support alternative distros/repos like Debian or Fedora, this may be less straight-forward and need some config support at that point to support specifying the appropriate mappings?

Raspbian apparently considers armhf as arm/v6, I've also seen on DockerHub where Debian and openSUSE Leap images archs differ a bit, linux/arm seems to resolve to the lowest supported variant, so if both armv6 and armv7 are published you'd get armv6 (openSUSE Leap is known for this with 15.5 at least), Debian I've seen only publishes linux/arm/v5 and linux/arm/v7, where pulling a linux/arm/v6 gets you the linux/arm/v5 platform image (which Raspbian also considers as armel).

@polarathene

Hi @polarathene
thanks for the detailed description.

Currently: Chisel's --arch expects deb archs, while the Chisel GH releases are names after Go archs

chisel cut --arch does not map TARGETARCH (or related ARG from BuildKit) to the repo architectures expected / supported.

I don't think this would change. We want the UX to be deb-derived, and the TARGETARCH is based on Go archs. So, you can actually use TARGETARCH to download the Chisel releases from GitHub.

I think I'm missing context on the user story that you're aiming for here. Let me try to depict a few different flows, and their implications

1. `chisel cut` for arch xyz, from a xyz Deb build environment, without buildkit (i.e TARGETARCH)

Since it's a Deb build environment, we can rely on arch="$(dpkg --print-architecture | sed -e 's/armhf/arm/g' -e 's/ppc64el/ppc64le/g')" to get the Go arch string that we'll use to download the GH release.

2. `chisel cut` for arch xyz, from a xyz NON-Deb build environment, without buildkit (i.e TARGETARCH)

This will be very distro-specific, but one might indeed need a custom mapping. Another option is to rely on Golang, if installed in the system, e.g. use go env GOARCH to determine the arch string to use when downloading the Chisel GH release.

3. `chisel cut` for arch xyz, from a xyz multi-arch build environment, WITH buildkit (i.e TARGETARCH)

Just rely on TARGETARCH. E.g. "https://github.com/canonical/chisel/releases/download/v${CHISEL_VERSION}/chisel_v${CHISEL_VERSION}_linux_${TARGETARCH}.tar.gz"

4. `chisel cut` for a foreign arch (i.e. an architecture that is different from the build env's)

4.1 without buildkit (i.e TARGETARCH)

This is the trickiest case, cause you'll need to know the deb arch you're aiming for, as well as its Go counterpart for downloading the Chisel GH release. So some sort of custom mapping is indeed necessary.

4.2 WITH buildkit (i.e TARGETARCH)

In this case, you still need some sort of Go-2-deb mapping, but we know that these arch strings only change in a few cases. This is the snippet you provided above, but simplified to cover only the diffs we know and support:

RUN <<HEREDOC
  case "${TARGETARCH}" in
    ( 'arm' )
      echo 'armhf' > /target-arch.txt 
      ;;
    ( '386' )
      echo 'i386' > /target-arch.txt
      ;;
    ( 'ppc64le' )
      echo 'ppc64el' > /target-arch.txt
      ;;
    ( * )
      echo "ERROR: Unsupported target platform: '${TARGETPLATFORM}'"
      return 1
      ;;
  esac
HEREDOC

I hope I captured the most common use cases. Is your use case different from these? If not, apart from 4.1, all the others seem quite low effort.

And yes, in a container context, all of this becomes easier with a Chisel container (fyi, an unofficial image in WIP: https://github.com/rockcrafters/chisel-rock by @rebornplusplus)

Apologies for the delayed response. I somehow missed the notification.

Currently: Chisel's --arch expects deb archs, while the Chisel GH releases are names after Go archs

I am aware of this now, but I do recall losing time trying to lookup the deb arch values and get that working properly for the more niche archs.

I don't think it's called "Go arch" specifically? (I know it's the arch convention Go uses, but it's not the only one to use that convention)

The deb archs was a bit more effort to grok and acquire a list / mapping of. If Chisel will not add support mapping/normalizing archs for --arch (or via some other flag/option/ENV), it would be helpful to document the supported archs that --arch expects.

These are not straight-forward if you're not familiar with them already, but are familiar with the Go arch's with BuildKit TARGETARCH and general linux uname -m being the two most common arch conventions.
Ubuntu additionally requires a bit more effort to add extra platform support if you're not building for the same arch it provides by default. Whereas Debian it was much simpler.

chisel cut --arch does not map TARGETARCH (or related ARG from BuildKit) to the repo architectures expected / supported.

I don't think this would change. We want the UX to be deb-derived, and the TARGETARCH is based on Go archs. So, you can actually use TARGETARCH to download the Chisel releases from GitHub.

Not all Go arch's are published for their default native Deb arch to be used.
A tool that is container focused but cannot map TARGETARCH convention to deb arch equivalent is poor UX for native build nodes that could otherwise be much faster for cutting a non-native arch.

1. chisel cut for arch xyz, from a xyz Deb build environment, without buildkit (i.e TARGETARCH)

Since it's a Deb build environment, we can rely on arch="$(dpkg --print-architecture | sed -e 's/armhf/arm/g' -e 's/ppc64el/ppc64le/g')" to get the Go arch string that we'll use to download the GH release.

I mean yeah that kinda works? But I'd also assume Chisel being container oriented will see more use in other build environments.

Is this debian build environment specific workaround officially documented, or do users need to find it through search? (or like in my case, think of more potable alternatives)

You'll still have very slow builds in comparison to native arch cutting foreign archs (when using CI to build for multiple archs), but when sed is available it could be leveraged for that too with buildkit.

2. chisel cut for arch xyz, from a xyz NON-Deb build environment, without buildkit (i.e TARGETARCH)

This will be very distro-specific, but one might indeed need a custom mapping. Another option is to rely on Golang, if installed in the system, e.g. use go env GOARCH to determine the arch string to use when downloading the Chisel GH release.

I don't know why this needs to be complicated? Platforms aren't likely to vary much here. If that's a concern, a file mapping could be used and one could be embedded by default (or similar internal code as fallback).

The point is it should not need to be distro specific. Even if chisel were to support other distros in the far future (beyond Ubuntu slices), that --arch option may take different values for equivalent archs, but common mapping/normalization logic for the intended arch would effectively be the same.

It'd be much better (and portable) for UX for chisel to have the functionality to do such a mapping internally (even if that invokes a separate flag like --from-go-arch).

3. chisel cut for arch xyz, from a xyz multi-arch build environment, WITH buildkit (i.e TARGETARCH)

Just rely on TARGETARCH. E.g. "https://github.com/canonical/chisel/releases/download/v${CHISEL_VERSION}/chisel_v${CHISEL_VERSION}_linux_${TARGETARCH}.tar.gz"

One of the first points I mentioned in my issue description was that GH releases does not cover all supported --arch:

You are missing 386 and AFAIK the arm variant is armhf, there is no armel.

At the end of my issue description I note how armhf / armel differs for different debian distros. I also note with respect to DockerHub base images published that this is inconsistent and thus care needs to be taken when bringing in arm image variants that aren't arm64.

All of those concerns can be avoided however by only using the image for your native build host (likely AMD64/ARM64) and providing chisel with --arch with correct mapping normalization in place. Not only is the much nicer and simpler to work with, but it is considerably faster than running emulated builds for non-native archs through QEMU.

4. chisel cut for a foreign arch (i.e. an architecture that is different from the build env's)

4.1 without buildkit (i.e TARGETARCH)

This is the trickiest case, cause you'll need to know the deb arch you're aiming for, as well as its Go counterpart for downloading the Chisel GH release. So some sort of custom mapping is indeed necessary.

4.2 WITH buildkit (i.e TARGETARCH)

In this case, you still need some sort of Go-2-deb mapping, but we know that these arch strings only change in a few cases. This is the snippet you provided above, but simplified to cover only the diffs we know and support:

It doesn't matter that only a subset changes, you still need to be consistent in the usage? (reading the arch in from a text file or ENV)

And yes, in a container context, all of this becomes easier with a Chisel container (fyi, an unofficial image in WIP: https://github.com/rockcrafters/chisel-rock by @rebornplusplus)

Is that published anywhere? It has had no activity since Oct 2024 and remains a draft PR: rockcrafters/chisel-rock#1

The author has commented in Feb 2025 in the related chisel discussion for an official image: #203 (comment)

Another user shared their own attempt at a chisel image and I provided feedback there, and earlier my own feedback on the topic with example Dockerfiles. I'll likely get around to just building and publishing my own Chisel image once some of my concerns are better addressed with improving Chisel.

As Chisel is not the only tool I run into issues with regarding mapping arch conventions from one to another, I'll probably just build a static binary that can do this on-demand.

This issue was mainly to raise awareness of the UX problems related to Chisel with a real-world example (testssl.sh) where this added a fair amount of friction when looking into using Chisel.

@letFunny

Hi, no worries. Some quick remarks:

A tool that is container focused but cannot map TARGETARCH convention to deb arch equivalent is poor UX for native build nodes that could otherwise be much faster for cutting a non-native arch.
...
... Chisel being container oriented will see more use in other build environments.

please keep in mind that Chisel isn't a container tool. Yes, is has been extremely useful for container builds, but at its core, it creates Ubuntu filesystems, that you can then package in any format you'd like, not just OCI.

chisel cut for arch xyz, from a xyz NON-Deb build environment, without buildkit (i.e TARGETARCH)
This will be very distro-specific, but one might indeed need a custom mapping. Another option is to rely on Golang, if installed in the system, e.g. use go env GOARCH to determine the arch string to use when downloading the Chisel GH release.

I don't know why this needs to be complicated? Platforms aren't likely to vary much here. If that's a concern, a file mapping could be used and one could be embedded by default (or similar internal code as fallback).

That's precisely what I mean - a mapping. Nothing complicated.

The point is it should not need to be distro specific. Even if chisel were to support other distros in the far future (beyond Ubuntu slices), that --arch option may take different values for equivalent archs, but common mapping/normalization logic for the intended arch would effectively be the same.
It'd be much better (and portable) for UX for chisel to have the functionality to do such a mapping internally (even if that invokes a separate flag like --from-go-arch).

In this case you'd just be shifting the architecture mapping into Chisel's codebase, which although possible, it would become a can of worms, as a single architecture can have a growing number of different notations if considering different environments/distros. The risk/benefit ratio isn't justified IMO, as the cost of doing said mapping outside Chisel is a very low. The bottom line is that Chisel works with Ubuntu packages, and for those, we rely on the debian architecture notation.

You are missing 386 and AFAIK the arm variant is armhf, there is no armel.

Indeed we don't build for 386 (intentionally). That also applied to our container images in Docker Hub.
In GitHub releases, we name the artifacts after the GOARCH, so armhf (the arm 32bit arch that we support) becomes arm (we don't take into account the GOARM value for naming the GH release assets).

Is that published anywhere? It has had no activity since Oct 2024 and remains a draft PR: rockcrafters/chisel-rock#1

It hasn't. Atm Chisel is delivered through GH releases and Snap Store.

Thanks for the detailed discussion.
Given it's length, let me try to create an oversimplified summary for future reference:

Summary

Today, Chisel offers a UX that is designed to rely on the host to infer the target Ubuntu Release and Architecture to install slices from.

For those running on a non-Debian distro, Chisel offers 2 CLI options: --release and --arch to allow users enough flexibility to compensate for the lack of system data such as /etc/lsb-release (which Chisel relies on to infer the chisel-release).

At the same time, Chisel is not a container specific tool, so it hasn't been designed to dynamically work with things like TARGETARCH.

The above also applies to the naming convention we use in the GitHub releases (based on GOARCH). In this case, users who consume Chisel from a GitHub release may need to manually map their working architecture into GOARCH in order to download the right binary from GitHub. We may be able to improve this experience though:

Offer Chisel via a snap (architectures are dynamically resolved)
Offer Chisel via a container
Rethink the GitHub release assets and their naming convention

For cases where the GH release is being used, we don't have any short term plans to embed any sort of custom cross-distro architecture mapping into Chisel.

fyi @letFunny

Cheers for the clarification. My mistake regarding Chisel as container image oriented, I must be recalling some Canonical blog posts I came across when first hearing about it that were focused on building OCI images and comparing to alternative images 😅

I think it'd be nice to make progress towards an official Chisel image, is there an outline for any blockers regarding that or just a lower priority to allocate time towards over existing tasks for Chisel? (I understand that there are still some UX improvements that could be done to make using Chisel less verbose, beyond what this issue discusses)

I'll close this issue as there isn't that much actionable for Chisel beyond acquiring a release.

I've only come across one project (testssl.sh) that wanted to retain support for what are now niche archs, and in their case several packages are lacking slices (which I have opened issues for and implemented myself), yet the final image for that project was effectively the same size as their existing openSUSE Leap image at 54MB (while the Alpine variant is 27MB).. so they have little incentive to switch given all that added friction without any benefits.

Thanks for the detailed responses ❤

Navigation Menu

Repository navigation

Better support building multi-arch `Dockerfile` via platform `ARG` mapping to `chisel cut --arch` #223

Problem and workarounds

1. `chisel cut` for arch xyz, from a xyz Deb build environment, without buildkit (i.e TARGETARCH)

2. `chisel cut` for arch xyz, from a xyz NON-Deb build environment, without buildkit (i.e TARGETARCH)

3. `chisel cut` for arch xyz, from a xyz multi-arch build environment, WITH buildkit (i.e TARGETARCH)

4. `chisel cut` for a foreign arch (i.e. an architecture that is different from the build env's)

4.1 without buildkit (i.e TARGETARCH)

4.2 WITH buildkit (i.e TARGETARCH)

1. `chisel cut` for arch xyz, from a xyz Deb build environment, without buildkit (i.e TARGETARCH)

2. `chisel cut` for arch xyz, from a xyz NON-Deb build environment, without buildkit (i.e TARGETARCH)

3. `chisel cut` for arch xyz, from a xyz multi-arch build environment, WITH buildkit (i.e TARGETARCH)

4. `chisel cut` for a foreign arch (i.e. an architecture that is different from the build env's)

4.1 without buildkit (i.e TARGETARCH)

4.2 WITH buildkit (i.e TARGETARCH)

Summary

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Better support building multi-arch Dockerfile via platform ARG mapping to chisel cut --arch #223

Description

Problem and workarounds

Activity

cjdcordeiro commented on May 6, 2025

1. chisel cut for arch xyz, from a xyz Deb build environment, without buildkit (i.e TARGETARCH)

2. chisel cut for arch xyz, from a xyz NON-Deb build environment, without buildkit (i.e TARGETARCH)

3. chisel cut for arch xyz, from a xyz multi-arch build environment, WITH buildkit (i.e TARGETARCH)

4. chisel cut for a foreign arch (i.e. an architecture that is different from the build env's)

4.1 without buildkit (i.e TARGETARCH)

4.2 WITH buildkit (i.e TARGETARCH)

polarathene commented on Jun 17, 2025

1. chisel cut for arch xyz, from a xyz Deb build environment, without buildkit (i.e TARGETARCH)

2. chisel cut for arch xyz, from a xyz NON-Deb build environment, without buildkit (i.e TARGETARCH)

3. chisel cut for arch xyz, from a xyz multi-arch build environment, WITH buildkit (i.e TARGETARCH)

4. chisel cut for a foreign arch (i.e. an architecture that is different from the build env's)

4.1 without buildkit (i.e TARGETARCH)

4.2 WITH buildkit (i.e TARGETARCH)

polarathene commented on Jun 17, 2025

cjdcordeiro commented on Jun 20, 2025

Summary

polarathene commented on Jun 21, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Better support building multi-arch `Dockerfile` via platform `ARG` mapping to `chisel cut --arch` #223

1. `chisel cut` for arch xyz, from a xyz Deb build environment, without buildkit (i.e TARGETARCH)

2. `chisel cut` for arch xyz, from a xyz NON-Deb build environment, without buildkit (i.e TARGETARCH)

3. `chisel cut` for arch xyz, from a xyz multi-arch build environment, WITH buildkit (i.e TARGETARCH)

4. `chisel cut` for a foreign arch (i.e. an architecture that is different from the build env's)

1. `chisel cut` for arch xyz, from a xyz Deb build environment, without buildkit (i.e TARGETARCH)

2. `chisel cut` for arch xyz, from a xyz NON-Deb build environment, without buildkit (i.e TARGETARCH)

3. `chisel cut` for arch xyz, from a xyz multi-arch build environment, WITH buildkit (i.e TARGETARCH)

4. `chisel cut` for a foreign arch (i.e. an architecture that is different from the build env's)