This looks like way too much extra work for a very custom behavior. I think instead, we should a "material-attestation" and that would save the snapshots. Currently, in provenance we have digests for the materials, but we could also have a separate manifest that would allow pulling all these digests from the registry.

Could you elaborate? How will the Dockerfile and the build command look like?

I think we've discussed this in the past. Maybe in slack. For alpine you would need something like https://github.com/tonistiigi/buildkit-alpine . Eg. this is an example image https://explore.ggcr.dev/?image=tonistiigi%2Ftest:alpine-curl . If you look at the provenance attestation then there are individual alpine packages all there with their checksums. The missing piece would be to also push the blobs to the registry and then you can use the source policy rule that converts from https URL or apk package into the same blob in registry.

I think we've discussed this in the past. Maybe in slack. For alpine you would need something like https://github.com/tonistiigi/buildkit-alpine .

Yes, but how is this expected to be integrated to the Dockerfile syntax?
I also wonder how we can make this portable to other package managers, while keeping the syntax as simple as possible.

Yes, but how is this expected to be integrated to the Dockerfile syntax?

You can use --build-context (or FROM) and longer term we need to figure out how to link frontends together with a nicer UI (maybe smth like FROM ./alpine.yml AS alpine)

I also wonder how we can make this portable to other package managers,

For anything that gets too custom in the package manger territory, it is easier to implement such things with full frontend API rather that let the user hack something together with Dockerfile limitations. We get lot of consistency for free like provenance materials detection and verification, unified storage and automatic snapshot loading from previous provenance via source policies. Doing this by hand doesn't have any of such guarantees/features.

Copying what we discussed in Slack here:

Example:

1. we make a buildkit release and push an image
2. a 3rd party downloads provenance for our release
3. it can now pull all the materials in that provenance and verify them offline if it wants to
4. it issues a rebuild with a special policy that doesn’t allow any outside images or new materials not in the policy. It can also drop the network completely.
5. it checks if it got the same artifact digest
6. if it did, then it issues a new (signed) attestation, confirming that our provenance was correct and is based only on our strictly defined sources
7. our image is now more trustworthy because trusted parties not related to us have verified it. It is impossible it was built from a a different Git commit or used apk package with a CVE for example.

Alternatively, we may just support OCI exporter for cache mounts and call it a day.

Dockerfile:

FROM alpine:3.18@sha256:deadbeef
RUN --mount=type=cache,target=/etc/apk/cache,id=etc-apk-cache \
  apk add gcc

Push:

# orasify=true ORAS-ifies the cache blobs for deduplication. 
# (1 blob = 1 apk file)
# This option should be optional.
docker buildx build . \
  --push \
  -t example.com/myimage:v1.2.3 \
  --cache-to type=oci,orasify=true,cache-mount-id=etc-apk-cache,target=example.com/myimage:v1.2.3-etc-apk-cache

Repro:

docker buildx build . \
  --cache-from type=oci,cache-mount-id=etc-apk-cache,src=example.com/myimage:v1.2.3-etc-apk-cache

Opened an issue to continue the discussion

• Attach package materials to an attestation for reproducible builds #4259