ociTools: rework OCI image building by msanft · Pull Request #390624 · NixOS/nixpkgs

msanft · 2025-03-17T10:13:16Z

OCI support in nixpkgs has been lacking, with only the long-obsolete v1 runtime specification, dated 2017, being supported.

This PR changes the OCI interface in nixpkgs completely, introducing 100%-pkgs.dockerTools-compatible buildImage and buildLayeredImage functions to build OCI directories or tarballs by transforming Docker images via skopeo into OCI images that conform to the v1.2 runtime specification.

It might not be the most efficient implementation (see commit descriptions for further rationale behind this), but the sheer complexity of dockerTools has lead me to believe that this is the best solution to get some kind of OCI image building support into nixpkgs in the short to mid term, and it removes the ugly footguns and incompatibilities of ociTools.buildContainer (the symptoms of which can be seen in the linked issues) in favor of a more coherent interface.

Closes #290879
Closes #290881
Closes #290882
Closes #293849
Closes #316652

Things done

Add a 👍 reaction to pull requests you find important.

pkgs/build-support/oci/default.nix

doc/redirects.json

nixos/doc/manual/release-notes/rl-2505.section.md

msanft · 2025-03-19T08:09:36Z

@fricklerhandwerk - as you worked on the docs for some time iirc:

NOTE: If your Manual build passes locally and you see this message in CI, you probably need a rebase.

The build indeed works locally, but I can't quite understand this notice. I'd understand it if there'd be sections that the remote state doesn't see, but that should not be the case here. Onto what should one rebase here to have the manual build succeed in CI?

fricklerhandwerk · 2025-03-19T11:25:29Z

@msanft you may need to rebase on top of master

msanft · 2025-03-19T12:08:28Z

you may need to rebase on top of master

Hmm. I think I should be.

fricklerhandwerk · 2025-03-20T08:37:57Z

@GetPsyched do you have a clue what's going on here? This should actually be green from what I can see.

philiptaron · 2025-06-05T23:32:36Z

I should have time to review this soon. Could you rebase and move the release notes to the next version of NixOS, @msanft? Thanks.

msanft · 2025-06-12T14:36:16Z

I should have time to review this soon. Could you rebase and move the release notes to the next version of NixOS, @msanft? Thanks.

Thanks - unfortunately only came to this now. Should be rebased now and ready for another round of review :)

h0nIg · 2025-06-20T06:29:32Z

fyi, streamlayeredimage works in combination with skopeo, you dont need to write the tar to the store:

      ${script-from-streamlayeredimage} | ${pkgs.skopeo}/bin/skopeo \
        --insecure-policy copy --dest-precompute-digests \
        ${compressionOpts} \
        docker-archive:/dev/stdin \
        docker://xxxxxxxx

nixos-discourse · 2025-08-28T15:56:07Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/steiger-oci-native-builds-and-deployments-for-nix-bazel-and-docker-with-direct-registry-push/68607/1

This aligns `ociTools` with `dockerTools` (and all other language-specific build support tools) in the nixpkgs tree by moving it from the `oci-tools` subdirectory to an `oci` subdirectory.

This introduces a generic `toOCIImage` conversion function that converts Docker-style tarballs to OCI directories or tarballs using `skopeo`. This allows for building OCI [v1.2](https://opencontainers.org/posts/blog/2024-02-18-oci-runtime-spec-v1-2/) images with in-tree nixpkgs tools. This introduces a separate derivation for OCI images, which - obviously - comes at a cost. However, I deemed that to be worth it here, due to the sheer complexity an OCI builder would have to implement to keep up with `dockerTools` otherwise.

This aligns the ociTools documentation with the refactoring to the Docker-based conversion builds.

drupol · 2025-11-08T22:11:57Z

pkgs/build-support/oci/default.nix

+  nixosTests,
+}:
+
+rec {


What would it take to get rid of this rec?

I stuffed toOCIImage and mkDockerToolsDropin in a let-binding now. Is that preferrable?

This adds a NixOS test that exercises the reworked OCI functionality by executing OCI images via Docker.

RossSmyth · 2025-11-13T15:43:45Z

pkgs/build-support/oci/default.nix

+          skopeoOutputFormats."${outputFormat}"
+            or (throw "`outputFormat` must be one of: ${lib.concatStringsSep ", " (lib.attrNames skopeoOutputFormats)}");
+      in
+      stdenvNoCC.mkDerivation {


Shouldn't put that function call here. The constructDrv attribute set above does this for you in a proper fixed-point way.

Suggested change

stdenvNoCC.mkDerivation {

{

RossSmyth · 2025-11-13T15:46:03Z

pkgs/build-support/oci/default.nix

+    constructDrv = stdenvNoCC.mkDerivation;
+    excludeDrvArgNames = [
+      "docker-tarball"
+      "name"


Once you do the other change this may need to be removed. This is because the name attribute probably does need to be passed down to the next drv constructor.

Suggested change

"name"

msanft requested a review from katexochen March 17, 2025 10:13

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation labels Mar 17, 2025

nix-owners bot requested a review from philiptaron March 17, 2025 10:14

msanft force-pushed the msanft/oci/refactor branch from 19913d7 to e3bba0c Compare March 17, 2025 10:16

github-actions bot added 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Mar 17, 2025

msanft force-pushed the msanft/oci/refactor branch from e3bba0c to d6cc4bc Compare March 17, 2025 11:41

nix-owners bot requested review from GetPsyched, fricklerhandwerk and hsjobeki March 17, 2025 11:43

drupol reviewed Mar 17, 2025

View reviewed changes

pkgs/build-support/oci/default.nix Outdated Show resolved Hide resolved

fricklerhandwerk reviewed Mar 17, 2025

View reviewed changes

doc/redirects.json Show resolved Hide resolved

fricklerhandwerk reviewed Mar 18, 2025

View reviewed changes

nixos/doc/manual/release-notes/rl-2505.section.md Outdated Show resolved Hide resolved

msanft force-pushed the msanft/oci/refactor branch from d6cc4bc to 63a6539 Compare March 19, 2025 07:55

ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 19, 2025

github-actions bot removed the 8.has: changelog This PR adds or changes release notes label Mar 19, 2025

msanft force-pushed the msanft/oci/refactor branch 2 times, most recently from 42e25b3 to 3c633cc Compare March 19, 2025 07:59

msanft requested review from drupol and fricklerhandwerk March 19, 2025 07:59

ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 19, 2025

msanft force-pushed the msanft/oci/refactor branch from 3c633cc to 37cf6cc Compare March 19, 2025 08:05

msanft force-pushed the msanft/oci/refactor branch from 37cf6cc to 0913f76 Compare March 19, 2025 12:03

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 9, 2025

msanft force-pushed the msanft/oci/refactor branch from 2227aea to 3a858b3 Compare June 12, 2025 14:35

ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 12, 2025

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 16, 2025

nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 25, 2025

nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Aug 10, 2025

juliamertz mentioned this pull request Aug 22, 2025

feat: nix builder brainhivenl/steiger#1

Merged

msanft force-pushed the msanft/oci/refactor branch from 3a858b3 to 06834ca Compare September 1, 2025 08:47

nixpkgs-ci bot added 8.has: changelog This PR adds or changes release notes 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Sep 1, 2025

msanft added 4 commits November 8, 2025 22:11

ociTools: move to oci directory

c01d2fa

This aligns `ociTools` with `dockerTools` (and all other language-specific build support tools) in the nixpkgs tree by moving it from the `oci-tools` subdirectory to an `oci` subdirectory.

doc: rework ociTools section

ad63c2a

This aligns the ociTools documentation with the refactoring to the Docker-based conversion builds.

nixos/doc: add OCI rework to release notes

51ff9d8

msanft force-pushed the msanft/oci/refactor branch from 06834ca to 23e6ff2 Compare November 8, 2025 21:24

nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 8, 2025

drupol reviewed Nov 8, 2025

View reviewed changes

msanft added 3 commits November 9, 2025 09:16

nixos/tests: add an OCI image test

568ee61

This adds a NixOS test that exercises the reworked OCI functionality by executing OCI images via Docker.

ociTools: use lib.extendMkDerivation

77af9e0

ociTools: rework superfluous recursion

69308a8

msanft force-pushed the msanft/oci/refactor branch from 23e6ff2 to 69308a8 Compare November 9, 2025 08:19

RossSmyth suggested changes Nov 13, 2025

View reviewed changes

nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 15, 2025

nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 29, 2025

nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Feb 1, 2026

Uh oh!

Conversation

msanft commented Mar 17, 2025

Things done

Uh oh!

Uh oh!

Uh oh!

Uh oh!

msanft commented Mar 19, 2025

Uh oh!

fricklerhandwerk commented Mar 19, 2025

Uh oh!

msanft commented Mar 19, 2025

Uh oh!

fricklerhandwerk commented Mar 20, 2025

Uh oh!

philiptaron commented Jun 5, 2025

Uh oh!

msanft commented Jun 12, 2025

Uh oh!

h0nIg commented Jun 20, 2025

Uh oh!

nixos-discourse commented Aug 28, 2025

Uh oh!

drupol Nov 8, 2025

Choose a reason for hiding this comment

Uh oh!

msanft Nov 9, 2025

Choose a reason for hiding this comment

Uh oh!

drupol Nov 9, 2025

Choose a reason for hiding this comment

Uh oh!

RossSmyth Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

RossSmyth Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants