Summary

The test-tier3 job in .github/workflows/kwok-recipes.yaml fails to start because its build matrix produces 288 configurations, exceeding GitHub Actions' hard maximum of 256 configurations per job.

.github/workflows/kwok-recipes.yaml (Line: 325, Col: 7):
Strategy for job 'test-tier3' produced 288 configurations which exceeds the maximum of 256 configurations

Impact

High — Tier 3 is the full-matrix backstop that runs on push-to-main and the nightly schedule (per ADR-003). When the matrix is rejected, the job never launches, so the full recipe × deployer coverage silently stops running. The summary job treats a non-failure/cancelled Tier 3 result as acceptable, so this can pass unnoticed while coverage is actually lost.

Root Cause

The Tier 3 matrix is a two-dimension cross-product (.github/workflows/kwok-recipes.yaml:326-328):

matrix:
  recipe: ${{ fromJSON(needs.discover.outputs.tier3) }}   # all testable overlays
  deployer: [helm, argocd-oci, argocd-helm-oci, flux-oci]  # 4 deployers

tier3 is the all set from the discover job — every overlay with a concrete service criterion. That set has grown to 72 recipes:

• 72 recipes × 4 deployers = 288 configurations > 256 cap.

The matrix has crossed the threshold organically as accelerator/service overlays were added; nothing in the workflow guards against it.

Steps to Reproduce

1. Push a change touching recipes/** (or the workflow itself) to main, or wait for the nightly 0 3 * * * schedule.
2. Observe the test-tier3 job fail to start with the "288 configurations exceeds the maximum of 256" error.

Expected Behavior

Tier 3 runs the intended recipe × deployer coverage without exceeding GitHub's matrix limit, and coverage scales as more overlays are added.

Actual Behavior

The matrix is rejected before any job runs; Tier 3 coverage does not execute.

Possible Fixes (for discussion)

1. Shard the matrix — split test-tier3 into N jobs each handling a slice of recipes (e.g., a recipe-shard dimension), keeping each job's cross-product under 256. Most robust against future growth.
2. Reduce deployer coverage per recipe — test all recipes against one canonical deployer (helm), and run the full 4-deployer matrix only against a representative subset.
3. Construct an explicit include: matrix in discover — emit {recipe, deployer} pairs as JSON and cap the count, so the limit is enforced at generation time rather than discovered at runtime.
4. Add a guard in discover — fail fast with a clear message (and/or auto-shard) when recipes × deployers would exceed 256, so the failure is actionable instead of opaque.

Option 1 or 3 is preferred since the recipe count will keep growing.

Additional Context

• Affected job: test-tier3 — .github/workflows/kwok-recipes.yaml:312-354
• Matrix definition: lines 324-328
• tier3 source (all set): discover job, lines 96-111 and 215-216
• The summary job (lines 394-398) only flags Tier 3 on failure/cancelled — a matrix-generation rejection may not surface as a hard failure, so this should be treated as a coverage gap, not just a red X.