Reproducible build failure compared to Podman #149
Description
Reproduction steps:
git clone [email protected]:awslabs/mcp.git
cd mcp/src/aws-documentation-mcp-server
container build -t awslabs/aws-documentation-mcp-server .
Output: (Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding)
peckn@80a9970ab2e5 Code % cd mcp/src/aws-documentation-mcp-server
peckn@80a9970ab2e5 aws-documentation-mcp-server % container build -t awslabs/aws-documentation-mcp-server .
[+] Building 37.8s (7/15)
=> [resolver] fetching image...public.ecr.aws/sam/build-python3.13@sha256:0df1547849c376aa21f9c726c0d71c4974415ed4405256f8d79f8c25716a9c63 0.0s
=> [resolver] fetching image...public.ecr.aws/sam/build-python3.13@sha256:0df1547849c376aa21f9c726c0d71c4974415ed4405256f8d79f8c25716a9c63 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 4.17MB 0.1s
=> oci-layout://public.ecr.aws/sam/build-python3.13@sha256:0df1547849c376aa21f9c726c0d71c4974415ed4405256f8d79f8c25716a9c63 10.9s
=> => resolve public.ecr.aws/sam/build-python3.13@sha256:0df1547849c376aa21f9c726c0d71c4974415ed4405256f8d79f8c25716a9c63 0.1s
=> => sha256:614b0f3af9d18a1e2847918ccf647bd83e47da6d3c7fbfa987901c17cff5015d 122.19kB / 122.19kB 0.0s
=> => sha256:e86f4a115b38f26ee3e1e6fda5b8e3322835f30d1f99b297a20874ebbea01caf 3.19MB / 3.19MB 0.1s
=> => sha256:67a6921b48bab4978da4e790de4a5ed80e056e557189ae5cbf608fccbc3c580a 4.38MB / 4.38MB 0.1s
=> => sha256:b105cb525794115cbcadda4fcc2c479293524dea523ae7d613ead86331d1e6b2 106.64MB / 106.64MB 1.6s
=> => sha256:b222774d42724f2e813fa998cda4955b3739864132a956c1b94323d41ea43623 1.82MB / 1.82MB 0.0s
=> => sha256:20891445ef2ec0766880f5bcf31bbaff1e05b32e8399315be6fda7b57ee252cf 5.21kB / 5.21kB 0.0s
=> => sha256:d96c25539da78d708d6698f729ad165c17ae92141021338ef42dd7647f4e11b3 5.72MB / 5.72MB 0.1s
=> => sha256:bfd20282ffd27c652fc1f075d7fdacb9e6496075e582e6f56c66edfa35c04b82 72.97MB / 72.97MB 1.2s
=> => sha256:4ae27b0b787413e7e6071ecff09754e5d043b5f01750b2bbbbb411409f0adb2b 234.33MB / 234.33MB 2.7s
=> => sha256:8334545c60bc7039ef18dd23b047a02fc2adad319ae52b941283a7f2bf76b7a4 60.87MB / 60.87MB 1.0s
=> => sha256:ef56379ca61a84980a59f72a1cee904f1c919676756d13e3d2aa5d350d32e1e5 14.19kB / 14.19kB 0.0s
=> => sha256:8e93fc704afb1a47fd03dce067a0c345a10a8ff17a4303b1d3e1988a368694c3 150.44MB / 150.44MB 1.5s
=> => sha256:ef05d9f2265b34e7a00e7aff623707cf81b4e793a99bdeb46a7b38d0974e944b 2.72MB / 2.72MB 0.0s
=> => sha256:90957be4b1b4b6f41dd4e6a276af049c0a3148020d3ca5246bf21a0998be2042 416B / 416B 0.0s
=> => sha256:05402c68f16c57aa5fb143820da6019a30bc5cbe784f9e72f17ab9365f03bd5b 88.12kB / 88.12kB 0.0s
=> => sha256:34328c8be51cbad703d9e17aaa048fa432339731a4eccf6e25a78162a439aa8f 37.90MB / 37.90MB 0.4s
=> => extracting sha256:34328c8be51cbad703d9e17aaa048fa432339731a4eccf6e25a78162a439aa8f 1.2s
=> => extracting sha256:05402c68f16c57aa5fb143820da6019a30bc5cbe784f9e72f17ab9365f03bd5b 0.0s
=> => extracting sha256:90957be4b1b4b6f41dd4e6a276af049c0a3148020d3ca5246bf21a0998be2042 0.0s
=> => extracting sha256:ef05d9f2265b34e7a00e7aff623707cf81b4e793a99bdeb46a7b38d0974e944b 0.0s
=> => extracting sha256:8e93fc704afb1a47fd03dce067a0c345a10a8ff17a4303b1d3e1988a368694c3 1.6s
=> => extracting sha256:ef56379ca61a84980a59f72a1cee904f1c919676756d13e3d2aa5d350d32e1e5 0.0s
=> => extracting sha256:b105cb525794115cbcadda4fcc2c479293524dea523ae7d613ead86331d1e6b2 1.0s
=> => extracting sha256:4ae27b0b787413e7e6071ecff09754e5d043b5f01750b2bbbbb411409f0adb2b 2.8s
=> => extracting sha256:8334545c60bc7039ef18dd23b047a02fc2adad319ae52b941283a7f2bf76b7a4 1.0s
=> => extracting sha256:bfd20282ffd27c652fc1f075d7fdacb9e6496075e582e6f56c66edfa35c04b82 1.2s
=> => extracting sha256:20891445ef2ec0766880f5bcf31bbaff1e05b32e8399315be6fda7b57ee252cf 0.0s
=> => extracting sha256:b222774d42724f2e813fa998cda4955b3739864132a956c1b94323d41ea43623 0.0s
=> => extracting sha256:d96c25539da78d708d6698f729ad165c17ae92141021338ef42dd7647f4e11b3 0.1s
=> => extracting sha256:e86f4a115b38f26ee3e1e6fda5b8e3322835f30d1f99b297a20874ebbea01caf 0.1s
=> => extracting sha256:67a6921b48bab4978da4e790de4a5ed80e056e557189ae5cbf608fccbc3c580a 0.1s
=> => extracting sha256:614b0f3af9d18a1e2847918ccf647bd83e47da6d3c7fbfa987901c17cff5015d 0.0s
=> [linux/arm64/v8 uv 1/7] WORKDIR /app 0.1s
=> ERROR [linux/arm64 stage-1 1/5] RUN dnf update -y && dnf install -y lsof && dnf clean all -y && rm -rf /var/cache/dnf && grou 0.1s
------
> [linux/arm64 stage-1 1/5] RUN dnf update -y && dnf install -y lsof && dnf clean all -y && rm -rf /var/cache/dnf && groupadd --force --system app && useradd app -g app -d /app && chmod o+x /root:
0.050 Python path configuration:
0.050 PYTHONHOME = (not set)
0.050 PYTHONPATH = (not set)
0.050 program name = '/usr/bin/python3'
0.050 isolated = 0
0.050 environment = 1
0.050 user site = 1
0.050 import site = 1
0.050 sys._base_executable = '/usr/bin/python3'
0.050 sys.base_prefix = '/usr'
0.050 sys.base_exec_prefix = '/usr'
0.050 sys.platlibdir = 'lib64'
0.050 sys.executable = '/usr/bin/python3'
0.050 sys.prefix = '/usr'
0.050 sys.exec_prefix = '/usr'
0.050 sys.path = [
0.050 '/usr/lib64/python39.zip',
0.050 '/usr/lib64/python3.9',
0.050 '/usr/lib64/python3.9/lib-dynload',
0.050 ]
0.050 Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
0.050 Python runtime state: core initialized
0.051 ValueError: source code string cannot contain null bytes
0.051
0.051 Current thread 0x0000ffff98c31020 (most recent call first):
0.051 <no Python frame>
------
Error: unknown (2): failed to solve: process "/bin/sh -c dnf update -y && dnf install -y lsof && dnf clean all -y && rm -rf /var/cache/dnf && groupadd --force --system app && useradd app -g app -d /app && chmod o+x /root" did not complete successfully: exit code: 1
Compare to:
podman build -t awslabs/aws-documentation-mcp-server .
Output:
[1/2] STEP 1/11: FROM public.ecr.aws/sam/build-python3.13@sha256:0df1547849c376aa21f9c726c0d71c4974415ed4405256f8d79f8c25716a9c63 AS uv
Trying to pull public.ecr.aws/sam/build-python3.13@sha256:0df1547849c376aa21f9c726c0d71c4974415ed4405256f8d79f8c25716a9c63...
Getting image source signatures
Copying blob sha256:ef56379ca61a84980a59f72a1cee904f1c919676756d13e3d2aa5d350d32e1e5
Copying blob sha256:05402c68f16c57aa5fb143820da6019a30bc5cbe784f9e72f17ab9365f03bd5b
Copying blob sha256:ef05d9f2265b34e7a00e7aff623707cf81b4e793a99bdeb46a7b38d0974e944b
Copying blob sha256:34328c8be51cbad703d9e17aaa048fa432339731a4eccf6e25a78162a439aa8f
Copying blob sha256:90957be4b1b4b6f41dd4e6a276af049c0a3148020d3ca5246bf21a0998be2042
Copying blob sha256:8e93fc704afb1a47fd03dce067a0c345a10a8ff17a4303b1d3e1988a368694c3
Copying blob sha256:b105cb525794115cbcadda4fcc2c479293524dea523ae7d613ead86331d1e6b2
Copying blob sha256:4ae27b0b787413e7e6071ecff09754e5d043b5f01750b2bbbbb411409f0adb2b
Copying blob sha256:8334545c60bc7039ef18dd23b047a02fc2adad319ae52b941283a7f2bf76b7a4
Copying blob sha256:bfd20282ffd27c652fc1f075d7fdacb9e6496075e582e6f56c66edfa35c04b82
Copying blob sha256:20891445ef2ec0766880f5bcf31bbaff1e05b32e8399315be6fda7b57ee252cf
Copying blob sha256:b222774d42724f2e813fa998cda4955b3739864132a956c1b94323d41ea43623
Copying blob sha256:d96c25539da78d708d6698f729ad165c17ae92141021338ef42dd7647f4e11b3
Copying blob sha256:e86f4a115b38f26ee3e1e6fda5b8e3322835f30d1f99b297a20874ebbea01caf
Copying blob sha256:67a6921b48bab4978da4e790de4a5ed80e056e557189ae5cbf608fccbc3c580a
Copying blob sha256:614b0f3af9d18a1e2847918ccf647bd83e47da6d3c7fbfa987901c17cff5015d
Copying config sha256:a80c46d46bb11ca603614b39626c24f5d7d20f825496c27f08ac4933fa830a60
Writing manifest to image destination
[1/2] STEP 2/11: WORKDIR /app
--> b4b2ff750f72
[1/2] STEP 3/11: ENV UV_COMPILE_BYTECODE=1
--> 64237c994bb3
[1/2] STEP 4/11: ENV UV_LINK_MODE=copy
--> a168d4c4daf4
[1/2] STEP 5/11: ENV UV_PYTHON_PREFERENCE=only-system
--> 593a3a69af85
[1/2] STEP 6/11: ENV UV_FROZEN=true
--> 1836f261d365
[1/2] STEP 7/11: COPY pyproject.toml uv.lock ./
--> 1d6f928827b3
[1/2] STEP 8/11: RUN --mount=type=cache,target=/root/.cache/uv pip install uv==0.7.11 && uv sync --frozen --no-install-project --no-dev --no-editable
Collecting uv==0.7.11
Downloading uv-0.7.11-py3-none-manylinux_2_28_aarch64.whl.metadata (11 kB)
Downloading uv-0.7.11-py3-none-manylinux_2_28_aarch64.whl (16.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.6/16.6 MB 6.8 MB/s eta 0:00:00
Installing collected packages: uv
Successfully installed uv-0.7.11
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
[notice] A new release of pip is available: 25.0.1 -> 25.1.1
[notice] To update, run: pip install --upgrade pip
Using CPython 3.13.3 interpreter at: /var/lang/bin/python3
Creating virtual environment at: .venv
Downloading pygments (1.2MiB)
Downloading pydantic-core (1.8MiB)
Downloading pydantic-core
Downloading pygments
Prepared 31 packages in 191ms
Installed 31 packages in 13ms
Bytecode compiled 972 files in 201ms
+ annotated-types==0.7.0
+ anyio==4.9.0
+ beautifulsoup4==4.13.4
+ certifi==2025.1.31
+ click==8.1.8
+ h11==0.14.0
+ httpcore==1.0.7
+ httpx==0.28.1
+ httpx-sse==0.4.0
+ idna==3.10
+ loguru==0.7.3
+ markdown-it-py==3.0.0
+ markdownify==1.1.0
+ mcp==1.6.0
+ mdurl==0.1.2
+ pydantic==2.11.4
+ pydantic-core==2.33.2
+ pydantic-settings==2.8.1
+ pygments==2.19.1
+ python-dotenv==1.1.0
+ rich==14.0.0
+ shellingham==1.5.4
+ six==1.17.0
+ sniffio==1.3.1
+ soupsieve==2.6
+ sse-starlette==2.2.1
+ starlette==0.46.1
+ typer==0.15.2
+ typing-extensions==4.13.0
+ typing-inspection==0.4.0
+ uvicorn==0.34.0
--> c3c666e7530a
[1/2] STEP 9/11: COPY . /app
--> fdf4c2633c19
[1/2] STEP 10/11: RUN --mount=type=cache,target=/root/.cache/uv uv sync --frozen --no-dev --no-editable
Building awslabs-aws-documentation-mcp-server @ file:///app
Built awslabs-aws-documentation-mcp-server @ file:///app
Prepared 1 package in 500ms
Installed 1 package in 0.94ms
Bytecode compiled 980 files in 36ms
+ awslabs-aws-documentation-mcp-server==1.0.2 (from file:///app)
--> 8f917030d4a2
[1/2] STEP 11/11: RUN mkdir -p /root/.local
--> cb1ebe298f83
[2/2] STEP 1/9: FROM public.ecr.aws/sam/build-python3.13@sha256:0df1547849c376aa21f9c726c0d71c4974415ed4405256f8d79f8c25716a9c63
[2/2] STEP 2/9: ENV PATH="/app/.venv/bin:$PATH:/usr/sbin"
--> 66ac270066dd
[2/2] STEP 3/9: RUN dnf update -y && dnf install -y lsof && dnf clean all -y && rm -rf /var/cache/dnf && groupadd --force --system app && useradd app -g app -d /app && chmod o+x /root
Failed to set locale, defaulting to C.UTF-8
Amazon Linux 2023 repository 14 MB/s | 35 MB 00:02
Last metadata expiration check: 0:00:05 ago on Wed Jun 11 19:19:27 2025.
Dependencies resolved.
Nothing to do.
Complete!
Failed to set locale, defaulting to C.UTF-8
Last metadata expiration check: 0:00:05 ago on Wed Jun 11 19:19:27 2025.
Dependencies resolved.
================================================================================
Package Architecture Version Repository Size
================================================================================
Installing:
lsof aarch64 4.94.0-1.amzn2023.0.2 amazonlinux 237 k
Installing dependencies:
libtirpc aarch64 1.3.3-0.amzn2023 amazonlinux 96 k
Transaction Summary
================================================================================
Install 2 Packages
Total download size: 333 k
Installed size: 987 k
Downloading Packages:
(1/2): lsof-4.94.0-1.amzn2023.0.2.aarch64.rpm 2.3 MB/s | 237 kB 00:00
(2/2): libtirpc-1.3.3-0.amzn2023.aarch64.rpm 579 kB/s | 96 kB 00:00
--------------------------------------------------------------------------------
Total 335 kB/s | 333 kB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : libtirpc-1.3.3-0.amzn2023.aarch64 1/2
Installing : lsof-4.94.0-1.amzn2023.0.2.aarch64 2/2
Running scriptlet: lsof-4.94.0-1.amzn2023.0.2.aarch64 2/2
Verifying : libtirpc-1.3.3-0.amzn2023.aarch64 1/2
Verifying : lsof-4.94.0-1.amzn2023.0.2.aarch64 2/2
Installed:
libtirpc-1.3.3-0.amzn2023.aarch64 lsof-4.94.0-1.amzn2023.0.2.aarch64
Complete!
Failed to set locale, defaulting to C.UTF-8
9 files removed
--> ef0945fcdb81
[2/2] STEP 4/9: COPY --from=uv --chown=app:app /root/.local /root/.local
--> 709fa079e4ff
[2/2] STEP 5/9: COPY --from=uv --chown=app:app /app/.venv /app/.venv
--> b32374d8e414
[2/2] STEP 6/9: COPY ./docker-healthcheck.sh /usr/local/bin/docker-healthcheck.sh
--> 69fd2aa4e853
[2/2] STEP 7/9: USER app
--> d3a0c4739739
[2/2] STEP 8/9: HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 CMD [ "docker-healthcheck.sh" ]
time="2025-06-12T07:19:35+12:00" level=warning msg="HEALTHCHECK is not supported for OCI image format and will be ignored. Must use `docker` format"
--> e0e8cbb3f45a
[2/2] STEP 9/9: ENTRYPOINT ["awslabs.aws-documentation-mcp-server"]
[2/2] COMMIT awslabs/aws-documentation-mcp-server
time="2025-06-12T07:19:35+12:00" level=warning msg="HEALTHCHECK is not supported for OCI image format and will be ignored. Must use `docker` format"
--> 0b3c5104ff01
Successfully tagged localhost/awslabs/aws-documentation-mcp-server:latest
0b3c5104ff016eca483f022135d0d00ebfc15f4fc56c7cf9d2de7a4460ea6ada