Parent: #1264 · Labels: enhancement, area/ci (auto, .github/**),
area/infra (auto, infra/**); adds tools/uat-broker/** labeler.yml glob +
dedicated CODEOWNERS entry if a Go helper is introduced · Size: M ·
Depends on: none

Goal. Own the day/night operating cycle and arbitrate scarce reserved GPU time so
contending runs queue instead of hard-failing, keyed by reservation and driven by a
checked-in reservation registry that new infra extends by data edit. The broker is the
single owner of: (a) the nightly UAT-batch state machine
(provision→CUJ→evidence→publish→teardown→next, sequential per cloud); (b) the
version-matrix schedule (time-boxed, main-first, then the previous N stable
releases, descending); (c) the daytime human-access lease (DC8) granted at the
morning handoff and released at the evening teardown; and (d) ad-hoc human/CI GPU
requests. The default cadence is nightly-batch + daytime-lease; ad-hoc dispatch is
still supported but is the exception, not the norm. Keep it boring — a lease record +
GH-Actions concurrency, no daemon.

Scope.

• Implement the day/night cycle as the scheduling contract: a nightly batch entry
  point that walks the version-matrix schedule sequentially per cloud (time-boxed,
  main-first, oldest releases dropped first when the window is tight), and a
  morning-handoff step that grants the daytime DC8 lease once the batch drains. Both
  the batch and the lease acquire/release the same reservation lease so CI and
  human use never overlap on one reservation.
• Add a reservation registry, infra/uat/reservations.yaml (sibling of the
  existing infra/uat-aws-account/, infra/uat-gcp-account/), enumerating rows
  {name, cloud, reservation-id, accelerator, gpu-count, cluster-config-path, test-config-dir}. Seed with the two known rows: AWS H100
  (cr-0cbe491320188dfa6, tests/uat/aws/cluster-config.yaml) and GCP H100
  (h100-a3-mega-phase-1-…, tests/uat/gcp/cluster-config.yaml).
• Implement the lease as a GitHub Actions concurrency group keyed by the
  reservation name (e.g. concurrency.group: uat-${{ inputs.reservation }},
  cancel-in-progress: false), replacing the current per-workflow group
  (uat-aws.yaml:35-37) that only serializes a workflow against itself. A run that
  targets a busy reservation waits behind the in-progress run. Document the
  one-in-progress-plus-one-pending limit of GH concurrency (a third contender
  supersedes the pending one) and that this is acceptable at launch (two
  reservations, at most cron + one human dispatch each); cross-reference the
  Deferred "standing broker service" pull-trigger as the escalation path for
  N-deep queueing.
• Surface a superseded run. Add a step that detects a concurrency cancellation
  (a third contender cancelling a pending run) and emits a clear, visible "run
  superseded" notice — or records the dropped request in the lease record — so a
  silently-cancelled scheduled run is never invisible to its requester. A
  concurrency-cancelled run is not a job failure, so without this the third
  requester gets no signal; tie this to DC6's failure-surfacing.
• A thin dispatch surface so a human or CI can request a run: a
  workflow_dispatch input reservation (+ intent, + optional aicr_version) on a
  shared uat-run.yaml, or a reusable workflow_call the per-cloud workflows invoke.
  Keep it boring — no new daemon, no external state store. The nightly batch and the
  daytime DC8 lease are the two scheduled callers of this same surface.
• Schedule + version-matrix expansion. The nightly batch entry point expands the
  version-matrix schedule into ordered cells (main first, then the previous N stable
  releases descending) and runs them sequentially per cloud until the time-box closes;
  when it closes, in-flight cells finish and remaining cells are dropped oldest-release
  first. The per-cell aicr_version is threaded through to DC2's install step (DC5
  owns version-parameterized install). The morning-handoff step then acquires the DC8
  daytime lease on each freed reservation.
• Document the model in docs/contributor/uat.md (new): the day/night cycle (night
  batch → morning handoff → day human-use → evening teardown), how to request GPU time,
  how queuing behaves, how a superseded run is reported, how the version-matrix schedule
  is time-boxed and ordered, and how to add a reservation row. Register the new page
  in docs/index.yml under the Contributor Guide section in the same PR (precedent:
  ADR-009 registers docs/user/recipe-health.md); otherwise the page is orphaned from
  the rendered Fern sidebar.

Out of scope. Per-intent cluster shaping and the daytime-deployment lifecycle / the
enforced cleanup-before-batch boundary (DC2); the daytime deployment's workload and
endpoint (DC8); the version-parameterized install mechanics (DC5); changing what runs on
the cluster (DC3). Cost accounting (deferred).

Key files. infra/uat/reservations.yaml (new), .github/workflows/uat-aws.yaml
(concurrency :35-37, capacity check :122-147, cron :29),
.github/workflows/uat-gcp.yaml (concurrency; cron :29; no capacity check
exists), a new shared uat-run.yaml, docs/contributor/uat.md, docs/index.yml.

Tests. A workflow-level test (or actionlint + a documented manual matrix) that
two dispatches on the same reservation serialize and the second does not error on a
busy reservation, and a third is reported as superseded rather than silently
cancelled; that the nightly-batch expansion orders cells main-first and drops the
oldest release first when the time-box is exceeded; a unit test on any Go registry /
schedule parser if introduced (75% floor on that package). actionlint is already a
merge-gate job (merge-gate.yaml:199) and must stay green.

Acceptance. The day/night cycle is owned by the broker (nightly batch → morning
DC8 lease → evening teardown), the version-matrix schedule is time-boxed and
main-first; contending runs on one reservation queue rather than hard-fail; a third
contender is explicitly reported as superseded; the reservation set is data-driven; the
GH-concurrency one-pending limit is documented; the model is documented and the page is
registered in docs/index.yml; actionlint green.

Child of #1264 · staged from docs/design/011-uat-dynamic-clusters.md (DC1).