Summary

Investigate replacing the chainsaw CLI binary in the deployment
validator image with an in-process Go executor for the
assert/error allowlist subset. The goal is a pure-Go validator
image built by ko (no Dockerfile, no GitHub-release fetch, no
sha256 dance, no .settings.yaml build-arg plumbing), with the
read-only allowlist becoming structural rather than a YAML
validator.

Background

PR #1235 (closed #1220) ships the chainsaw binary in the deployment
validator image via a multi-stage Dockerfile that downloads + checksums
the upstream release tarball. Two dispatch paths exist in
validators/chainsaw/runner.go:

1. Raw K8s YAML asserts → assertRawResources →
   github.com/kyverno/chainsaw/pkg/engine/checks (in-process Go
   library). No binary needed; ko would handle this path today.
2. Chainsaw Test format (apiVersion: chainsaw.kyverno.io/v1alpha1,
   kind: Test) → runChainsawBinary → external chainsaw CLI. This
   is what the Dockerfile multi-stage exists for.

After #1235 lands, registry-declared content is restricted to a
read-only allowlist (assert and error operations only). That means
the in-process executor would only need to drive those two ops, not
the full chainsaw operation set (script, apply, wait, catch,
finally, ...). Significantly smaller surface than reimplementing
chainsaw.

Proposal

Write a runChainsawTestInProcess shim that:

1. Unmarshals the Test YAML to v1alpha1.Test (already done by
   ValidateTestReadOnly in validators/chainsaw/allowlist.go).
2. Walks test.Spec.Steps[].Try[], dispatching each assert /
   error to
   github.com/kyverno/chainsaw/pkg/engine/checks.Check (the same
   library assertRawResources already uses).
3. Applies a retry loop with defaults.AssertRetryInterval until the
   step passes or the per-step timeout from defaults.ChainsawAssertTimeout
   elapses.
4. Returns the same chainsaw.Result shape as runChainsawBinary.

Acceptance Criteria

• New runChainsawTestInProcess function in validators/chainsaw/
  with parity tests covering every registry check (use the existing
  TestValidateTestReadOnly_RegistryContent corpus as the source).
• Output parity proven against the binary path on all in-tree
  recipes/checks/*/health-check.yaml files — same Result.Passed
  / Result.Error for the same input + cluster state.
• runChainsawBinary deleted; ChainsawBinary interface +
  NewChainsawBinary removed.
• validators/deployment/Dockerfile deleted; deployment validator
  switched to ko (matches cmd/aicr build pattern in the Makefile).
• Makefile image-validators target collapses the chainsaw build-args
  plumbing (no more CHAINSAW_VERSION / per-arch checksums for the
  deployment image).
• .settings.yaml testing_tools.chainsaw and chainsaw_checksums
  remain (still used by tools/setup-tools and make check-health),
  but no longer flow into image build args.
• Read-only allowlist enforcement collapses into structural shape:
  there is no code path in the validator image that can script or
  apply (the in-process executor only implements assert/error).
  ValidateTestReadOnly may remain as a defense-in-depth fast-fail
  before the executor runs, or be dropped if the executor's
  type-switch is itself the gate.
• Image size measured before / after; target ~30-40MB pure-Go vs
  the current ~80MB+ with chainsaw binary.

Trade-offs

Pros:

• Smaller image, no Dockerfile, no fetch / checksum dance.
• Read-only allowlist becomes structural — the validator image
  cannot invoke state-changing chainsaw ops because the code to
  do so isn't compiled in.
• ko build matches the rest of the AICR Go binary build path.
• No GitHub-release dependency at image build time.

Cons:

• Takes ownership of a chainsaw-Test-subset executor. When
  kyverno/chainsaw bumps semantics for assert (e.g., new
  operators, evaluator behavior changes), AICR must track it.
• Spike work for the parity test corpus; semantics-equivalence is
  the load-bearing argument, not just type-equivalence.

Open Questions

• Does the kyverno/chainsaw pkg/engine/checks API surface change
  often enough that tracking it is meaningful work, or is it stable
  enough to vendor-pin against?
• Is there a stable upstream pkg/runner/ (or equivalent) that
  exposes a Test-step executor we could call directly, instead of
  writing our own walker? Worth a quick survey before spike.
• Should the spike target assert/error only, or also accept
  description/timeouts metadata that exists on
  v1alpha1.Operation?

Sequencing

Independent of the #660 epic (#1219–#1223). Land after #1235 so the
binary-shipping baseline exists to measure parity against. The
allowlist lint guard (#1223) is independent and can land either
before or after this.

Related

• PR feat(validator): ship chainsaw binary; activate deployment-phase runner #1235 (closes Ship chainsaw binary + wire deployment-phase runner (#660 PR 2) #1220) — ships the chainsaw binary that this issue
  would later replace
• Ship chainsaw binary + wire deployment-phase runner (#660 PR 2) #1220 — image-build PR this builds on
• [EPIC] Reuse Chainsaw health checks in aicr validate --phase deployment #660 — epic