Skip to content

docker build does not create reproducible output (even with fixed SOURCE_DATE_EPOCH) #43124

Description

@sdettmer

I came here from distribution/distribution#3557, which is the first and only reasonable match for "docker bug report" on google. There I was pointed to write here and I hope I'm at the right place now. Otherwise could you please tell where to submit correctly?

Description

I'm failing to build a reproducible container. As a minimal example, I try:

FROM  scratch
LABEL maintainer="DETTMER Steffen <[email protected]>"
ADD Dockerfile /dummy.txt

but I get different output of docker save on each machine (or each try without cache). SOURCE_DATE_EPOCH seems not to work either (e.g. SOURCE_DATE_EPOCH=1 docker build --no-cache -t tag1 reproducible/).

How do I use docker to build reproducibly?

Steps to reproduce the issue:

Example session demonstrating the problem:

sdettmer@RefVm5:~/work/sash (sash * u=) $ cat reproducible/Dockerfile 
FROM  scratch
LABEL maintainer="DETTMER Steffen <[email protected]>"
ADD Dockerfile /dummy.txt
sdettmer@RefVm5:~/work/sash (sash * u=) $ SOURCE_DATE_EPOCH=1 docker build --no-cache -t tag1 reproducible/
Sending build context to Docker daemon  14.85kB
Step 1/3 : FROM  scratch
 ---> 
Step 2/3 : LABEL maintainer="DETTMER Steffen <[email protected]>"
 ---> Running in ceb079a6bf1f
Removing intermediate container ceb079a6bf1f
 ---> 4a8c034d964d
Step 3/3 : ADD Dockerfile /dummy.txt
 ---> f9a0a2711219
Successfully built f9a0a2711219
Successfully tagged tag1:latest
sdettmer@RefVm5:~/work/sash (sash * u=) $ docker save tag1:latest > try1.tar
sdettmer@RefVm5:~/work/sash (sash * u=) $ SOURCE_DATE_EPOCH=1 docker build --no-cache -t tag1 reproducible/
Sending build context to Docker daemon  14.85kB
Step 1/3 : FROM  scratch
 ---> 
Step 2/3 : LABEL maintainer="DETTMER Steffen <[email protected]>"
 ---> Running in 5711f601458c
Removing intermediate container 5711f601458c
 ---> bbb9ae94a041
Step 3/3 : ADD Dockerfile /dummy.txt
 ---> 3d05c8211784
Successfully built 3d05c8211784
Successfully tagged tag1:latest
sdettmer@RefVm5:~/work/onit/ref-env (sash * u=) $ docker save tag1:latest > try2.tar
sdettmer@RefVm5:~/work/onit/ref-env (sash * u=) $ diff try{1,2}.tar
Binary files try1.tar and try2.tar differ
sdettmer@RefVm5:~/work/onit/ref-env (sash * u=) $ 

Describe the results you received:

Binary files try1.tar and try2.tar differ
(the output contains timestamps that change).

Describe the results you expected:

I expect to be able to generate the same content later when using exactly the same input.

Can I use docker to build reproducibly?

Additional information you deem important (e.g. issue happens only occasionally):

I understand that this might not be an normally important use case and that many don't need this. I understand that of course I have to manage own source repositories to be able to include exactly the same content to the image. I know that producing binary equal packages is hard (I already spent many working hours on that in different packages, with and without success).

Output of docker version:

Client:
 Version:           20.10.5+dfsg1
 API version:       1.41
 Go version:        go1.15.9
 Git commit:        55c4c88
 Built:             Wed Aug  4 19:55:57 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.5+dfsg1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.15.9
  Git commit:       363e9a8
  Built:            Wed Aug  4 19:55:57 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.5~ds1
  GitCommit:        1.4.5~ds1-2+deb11u1
 runc:
  Version:          1.0.0~rc93+ds1
  GitCommit:        1.0.0~rc93+ds1-5+b2
 docker-init:
  Version:          0.19.0
  GitCommit:      

Output of docker info:

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 23
  Running: 0
  Paused: 0
  Stopped: 23
 Images: 22
 Server Version: 20.10.5+dfsg1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1.4.5~ds1-2+deb11u1
 runc version: 1.0.0~rc93+ds1-5+b2
 init version: 
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.10.0-9-amd64
 Operating System: Devuan GNU/Linux 4 (chimaera)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 1.935GiB
 Name: RefVm5
 ID: DNAY:PMGH:GW2F:4Z4Y:HHXQ:3WTP:J4RO:DXOI:DNUK:U7KH:VF7P:LZWG
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

My tests where performed on a Devuan Chimaera 4.0 (a Debian Bullseye 11.1 based GNU Linux distribution) based proprietary virtual machine running in VirtualBox, but I'm quite sure that this is a generic issue and I'm afraid support for "physical" reproducible builds is not yet included in docker, but I think it is well suited, because docker already is a tool for "logical" repducible builds and due to the cache, a similar but not reliable effect already may happen in practice (i.e. the rebuilt packages on first glace look reproducible when built on same node due to the chache).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/builderBuildkind/enhancementEnhancements are not bugs or new features but can improve usability or performance.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions