Repository navigation
feat: support Ubuntu Chiseled image patching - #1667
Merged
Merged
Conversation
Signed-off-by: Sertac Ozercan <[email protected]>
Signed-off-by: Sertac Ozercan <[email protected]>
Signed-off-by: Sertac Ozercan <[email protected]>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds first-class Ubuntu Chiseled image patching across the CLI, BuildKit frontend, bulk mode, and OCI export paths.
Changes:
- Supports apt-less full-status images and native
manifest.wallre-cuts. - Adds Chisel release selection, provenance, validation, and architecture handling.
- Adds tooling publication, integration/e2e coverage, fixtures, and documentation.
Show a summary per file
| File | Description |
|---|---|
README.md |
Documents Chiseled support. |
go.mod |
Adds manifest dependencies. |
go.sum |
Records dependency checksums. |
website/sidebars.js |
Adds the Chiseled guide. |
website/docs/chiseled-images.md |
Defines behavior and limitations. |
website/docs/scanner-plugins.md |
Documents Trivy limitations. |
website/docs/quick-start.md |
Adds Chiseled guidance. |
website/docs/multiplatform-patching.md |
Updates platform behavior. |
website/docs/introduction.md |
Describes comprehensive updates. |
website/docs/faq.md |
Updates Ubuntu tooling selection. |
website/docs/development-tips.md |
Documents Chiseled tests. |
website/docs/bulk-image-patching.md |
Documents bulk configuration. |
website/docs/buildkit-frontend.md |
Documents frontend options. |
pkg/types/options.go |
Adds Chisel release options. |
pkg/types/options_test.go |
Tests option propagation. |
pkg/chisel/arch.go |
Maps OCI architectures. |
pkg/chisel/arch_test.go |
Tests architecture mapping. |
pkg/chisel/release.go |
Parses and infers releases. |
pkg/chisel/release_test.go |
Tests release validation. |
pkg/chisel/manifest.go |
Parses bounded native manifests. |
pkg/chisel/manifest_test.go |
Tests manifest validation. |
pkg/pkgmgr/pkgmgr.go |
Adds manager options and metadata. |
pkg/pkgmgr/pkgmgr_test.go |
Tests manager metadata. |
pkg/pkgmgr/dpkg.go |
Classifies and patches DPKG layouts. |
pkg/pkgmgr/dpkg_test.go |
Tests DPKG behavior. |
pkg/pkgmgr/chisel.go |
Implements native re-cuts. |
pkg/pkgmgr/chisel_test.go |
Tests Chisel helpers. |
pkg/pkgmgr/scripts/apt_get_download.sh |
Installs external DPKG updates. |
pkg/pkgmgr/scripts/select_dpkg_updates.sh |
Selects comprehensive updates. |
pkg/pkgmgr/scripts/probe_dpkg.sh |
Probes target metadata. |
pkg/pkgmgr/scripts/finalize_dpkg_status.sh |
Restores metadata layout. |
pkg/pkgmgr/testdata/dpkg_full_status.txt |
Adds full-status fixture data. |
pkg/patch/build.go |
Reuses authenticated solve options. |
pkg/patch/core.go |
Integrates Chisel managers and metadata. |
pkg/patch/core_test.go |
Tests core metadata behavior. |
pkg/patch/patch.go |
Handles explicit report platforms. |
pkg/patch/patch_test.go |
Tests platform resolution. |
pkg/patch/single.go |
Adds preflight and annotations. |
pkg/patch/single_test.go |
Tests descriptor annotations. |
pkg/frontend/frontend.go |
Propagates multiarch metadata. |
pkg/frontend/build.go |
Adds Chisel frontend support. |
pkg/frontend/build_test.go |
Tests frontend metadata and paths. |
pkg/frontend/options.go |
Parses release options. |
pkg/frontend/options_test.go |
Tests option parsing. |
pkg/common/buildkit.go |
Adds labels and path inspection. |
pkg/common/buildkit_test.go |
Tests common BuildKit helpers. |
pkg/cmd/cmd.go |
Adds --chisel-release. |
pkg/cmd/cmd_test.go |
Tests flag validation. |
pkg/bulk/config.go |
Adds release configuration fields. |
pkg/bulk/config_test.go |
Tests bulk YAML parsing. |
pkg/bulk/engine.go |
Resolves release overrides. |
pkg/bulk/engine_test.go |
Tests override precedence. |
pkg/buildkit/buildkit.go |
Updates discovery and OCI metadata. |
pkg/buildkit/buildkit_test.go |
Tests discovery and export metadata. |
images/chisel/Dockerfile |
Builds Chisel tooling. |
images/chisel/validator/main.go |
Validates and reconciles filesystems. |
images/chisel/validator/main_test.go |
Tests filesystem reconciliation. |
integration/chisel/fixtures_test.go |
Validates integration fixtures. |
integration/chisel/fixtures/README.md |
Documents fixtures. |
integration/chisel/fixtures/real-images.json |
Records pinned image metadata. |
integration/chisel/fixtures/full-status-no-tools/Dockerfile |
Builds apt-less fixture. |
integration/chisel/fixtures/full-status-no-tools/sentinel.txt |
Tests unmanaged preservation. |
integration/chisel/fixtures/native-manifest-local/Dockerfile |
Builds native fixture. |
integration/chisel/fixtures/native-manifest-local/sentinel.txt |
Tests native preservation. |
integration/chisel/fixtures/manifest-schema-1.0/manifest.wall |
Adds compressed manifest fixture. |
integration/chisel/fixtures/manifest-schema-1.0/manifest.jsonwall |
Adds source manifest fixture. |
integration/chisel/fixtures/manifest-schema-1.0/expected.json |
Defines expected records. |
integration/chisel/fixtures/manifest-schema-1.0/data.txt |
Adds fixture content. |
integration/chisel/fixtures/manifest-schema-1.0/verify.sh |
Verifies fixture integrity. |
test/e2e/chisel/main_test.go |
Configures e2e tests. |
test/e2e/chisel/chisel_test.go |
Exercises real Chiseled images. |
test/e2e/chisel/README.md |
Documents e2e coverage. |
test/e2e/chisel/fixtures/test-images.json |
Pins real-image fixtures. |
.github/workflows/build.yml |
Adds Chiseled e2e CI. |
.github/workflows/publish-chisel-tooling-image.yml |
Publishes tooling images. |
.github/workflows/scripts/chisel/verify-image.sh |
Verifies published images. |
.github/workflows/scripts/chisel/ensure-tag-absent.sh |
Protects immutable tags. |
Review details
- Files reviewed: 75/77 changed files
- Comments generated: 6
- Review effort level: Balanced
Signed-off-by: Sertac Ozercan <[email protected]>
Contributor
There was a problem hiding this comment.
Review details
Suppressed comments (3)
pkg/pkgmgr/chisel.go:33
- The production patch path still executes a mutable tooling tag, despite the adjacent TODO and the image's own requirement to be consumed by immutable digest. A tag retarget would change the privileged Chisel/validator code used to construct patched images without changing Copa, creating a supply-chain risk; publish the manifest list and pin this reference to its digest before merging.
pkg/pkgmgr/scripts/apt_get_download.sh:61 apt-get downloadfetches only the named archives; it does not resolve dependencies. The preceding no-argument--fix-broken installonly evaluates the old status database, so a selected newer package that introduces or tightens a dependency is installed with--force-allwithout that dependency, leaving the output status inconsistent. Resolve the selected packages through a download-only install against the reconstructed status and install the resulting closure.
test/e2e/chisel/chisel_test.go:254- This expected set excludes the source index's attached
unknown/unknownattestation descriptors. Because preserved-platform export only copies architecture descriptors, the unchanged arm64, ppc64le, and s390x images lose their still-valid SBOM/provenance attestations in the mixed OCI layout. Preserve attestations whosevnd.docker.reference.digestpoints to an unchanged child and assert their descriptors and blobs here.
- Files reviewed: 77/79 changed files
- Comments generated: 1
- Review effort level: Balanced
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1667 +/- ##
==========================================
+ Coverage 45.23% 52.53% +7.29%
==========================================
Files 58 62 +4
Lines 10507 13361 +2854
==========================================
+ Hits 4753 7019 +2266
- Misses 5424 5677 +253
- Partials 330 665 +335
🚀 New features to boost your workflow:
|
Signed-off-by: Sertac Ozercan <[email protected]>
Signed-off-by: Sertac Ozercan <[email protected]>
Contributor
There was a problem hiding this comment.
Review details
Suppressed comments (3)
pkg/pkgmgr/chisel.go:33
- The production patch path still resolves the tooling image through a mutable tag. A tag can be moved after this release, causing the same Copa binary to execute different privileged tooling and emit misleading
v1.4.2provenance. Publish the multi-platform image first and pin this constant to its manifest-list digest before merging.
.github/workflows/build.yml:987 - This SHA is identified as
v2.19.4in the new publish workflow, so thev2.3.1annotation here is stale and misrepresents the pinned action version. Update the comment to keep dependency audits accurate.
uses: step-security/harden-runner@9af89fc71515a100421586dfdb3dc9c984fbf411 # v2.3.1
.github/workflows/build.yml:983
- The test command alone is allowed 55 minutes, while this 60-minute job also performs checkout, Docker restart, tool downloads, artifact extraction, and a source build of the tooling image. A valid slow 55-minute run can therefore be killed by the job timeout before completion. Increase the job timeout to leave setup/cleanup headroom.
timeout-minutes: 60
- Files reviewed: 77/79 changed files
- Comments generated: 2
- Review effort level: Balanced
Contributor
There was a problem hiding this comment.
Review details
Suppressed comments (3)
pkg/patch/core.go:120
- This fail-open path can patch a native Chisel image from a language-only report whenever the metadata stat fails, contradicting the documented rule that any report-driven native patch must be rejected before modification. Return the inspection error instead; only a successful negative probe should preserve language-only behavior.
pkg/common/os.go:92 strconv.Unquoteimplements Go string escapes, not theos-releasegrammar. Valid double-quoted values may escape$or backticks, which this rejects, while sequences such as\nare incorrectly decoded; because every assignment is parsed, an unrelated valid field can now prevent OS detection entirely. Restore the spec-aware parser or implement the documentedos-releaseescaping rules.
.github/workflows/publish-chisel-tooling-image.yml:21- The immutable
v1.4.2tag is already published and pinned elsewhere in this change, but adding this workflow andimages/chisel/**triggers it on the merge push tomain. Theensure-tag-absent.shstep will therefore fail immediately, so this publication job cannot succeed for the commit that introduces it. Publish under a new version/tag (updating all matching version and digest references), or make the workflow verify and reuse the already-published expected digest.
IMAGE_TAG: v1.4.2
- Files reviewed: 77/79 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Signed-off-by: Sertac Ozercan <[email protected]>
Member
Author
|
Also addressed the suppressed findings from the follow-up Copilot reviews in
Local validation passed: |
Member
Author
|
@codex review |
Contributor
There was a problem hiding this comment.
Review details
Suppressed comments (2)
pkg/pkgmgr/scripts/apt_get_download.sh:217
external-full-statusis selected when any required tool is missing, not only when all of these tools were absent. This unconditional cleanup can delete a target's pre-existing shell, apt, or dpkg binaries—for example, an otherwise normal image routed externally because onlyteeis missing. Track which tools were absent in the probe and remove only executables introduced by package expansion, preserving tools that existed in the source image.
.github/workflows/publish-chisel-tooling-image.yml:76- The PR states that the v1.4.2 tooling artifact has already been published, and this workflow also runs on the merge push because
images/chisel/**is changed. The guard will therefore find:v1.4.2and fail the first main-branch run, leaving the newly added publishing workflow red despite the existing artifact being the expected one. Make an existing tag a successful no-op only when it resolves to the expected digest, or avoid triggering this already-published version on push.
- name: Guard immutable release tag
env:
IMAGE_REF: ${{ env.IMAGE_NAME }}:${{ env.IMAGE_TAG }}
run: .github/workflows/scripts/chisel/ensure-tag-absent.sh "${IMAGE_REF}"
- Files reviewed: 77/79 changed files
- Comments generated: 3
- Review effort level: Balanced
Signed-off-by: Sertac Ozercan <[email protected]>
Signed-off-by: Sertac Ozercan <[email protected]>
Member
Author
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e2bf4eb806
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Sertac Ozercan <[email protected]>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2a73ac6942
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Sertac Ozercan <[email protected]>
Member
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c340385b63
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Sertac Ozercan <[email protected]>
Member
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d1dae7c990
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Sertac Ozercan <[email protected]>
Member
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. You're on a roll. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Signed-off-by: Sertac Ozercan <[email protected]>
Member
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Signed-off-by: Sertac Ozercan <[email protected]>
robert-cronin
approved these changes
Aug 25, 2026
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add first-class patching support for both Ubuntu Chiseled image layouts:
/var/lib/dpkg/statusbut lack target-side package-manager tooling. They now support both report-driven and comprehensive updates through external Ubuntu tooling while preserving the full-status representation./var/lib/chisel/manifest.wallnow support comprehensive re-cuts of every selected slice in a fresh staging root. Report-driven native patching is rejected with an actionable error.The change also adds:
--chisel-releasesupport across CLI, BuildKit frontend, and bulk configuration;Why
A full
/var/lib/dpkg/statusfile did not imply thatapt-get,dpkg, a shell, and the other required tools were available in the target image. Routing those images through the regular target-side path failed, while routing them through the existing status-directory path risked corrupting or replacing their package metadata.Native Chisel images have a different source of truth entirely: their package, slice, content, and path ownership state is recorded in
manifest.wall, so they require a complete re-cut rather than a synthetic dpkg database or targeted package merge.This PR supersedes #1634. Thanks to @AruneshDwivedi for the original problem report and initial investigation of apt-less Chiseled images.
User impact
manifest.wallimages are comprehensive-update-only.--chisel-releaseapplies only to native Chisel images; inference usesubuntu-<VERSION_ID>from/etc/os-release.linux/arm/v6.manifest.wall; native validation relies on Copa's manifest/filesystem checks and application smoke testing.Validation
make testmake lintwith the repository-pinned golangci-lint v2.10.1make -B buildgo test ./integration/chisel -count=1go test ./test/e2e/chisel -short -count=1TestNativeChiselRealImageTestNativeChiselCommunityImagePreservesApplicationTestAptlessFullStatusRealImageTestNativeChiselPartialPlatformOCIcd website && yarn buildactionlintsyntax/expression validation for the changed workflowsgofumptandgit diff --checkDraft follow-ups
ghcr.io/project-copacetic/copacetic/chisel@sha256:adc238182bcbc07ff5f030929732a46d7f1aab801fadb70b320805a1d56c817cDeferred
manifest.wallsupport.