Parent: #1263 · Labels: enhancement, area/infra (auto, GKE manifests
under infra/testgrid/**) · Size: L · Depends on: TG1, TG2 (needs a feed to
roll up and the pinned key schema to mirror)
Goal. Deploy the upstream Kubernetes TestGrid updater/tabulator/summarizer on
the eidosx host cluster and an AICR-native config generator that infers the
CSP-first taxonomy from the bucket layout, so the raw groups/ feed is rolled up
into the processed grid//tabs//summary/ trees the API consumes — with a
contract test that pins the TG2↔TG3 metadata-key seam.
Scope.
- Deploy the three upstream workers unchanged as GKE Workload Identity pods (on
the TG1 host cluster) with roles/storage.objectAdmin (the workers SA from TG1):
updater (30m, groups/→grid/), tabulator (15m, grid/→tabs/), summarizer
(5m, tabs/→summary/), each with --confirm, --wait, --persist-queue,
--json-logs, resumable queue/*.json. Manifests live under
infra/testgrid/k8s/ (or Helm/Kustomize as the repo prefers).
- AICR-native config-gen lives in its own tool:
tools/testgrid-config-gen/
(Go), owned by TG3 — it consumes the bucket layout the workers read, not the
OCI bundle TG2 reads, so it is a distinct artifact from TG2 with its own
labeler/CODEOWNERS/coverage scope. It lists gs://<bucket>/groups/, infers the
CSP-first taxonomy (DashboardGroup=service, Dashboard=accelerator-os,
TestGroup/Tab=intent[-platform]), and writes the protobuf Configuration to
gs://<bucket>/config. Honor the upstream invariants: GcsPrefix must include
the bucket name; each TestGroup must set result_source.gcs_config; configure
the AICR column-header set keyed to the exact metadata key strings TG2 emits
(the pinned schema) as the Header.extra array; alerting knobs
(numFailuresToAlert=1, alertStaleResultsHours=168, minAcceptableRuns=3,
brokenColumnThreshold=50%) as sane defaults; DisableProwjobAnalysis: true (no
podinfo.json).
- Run config-gen on a cadence (CI cron or an in-cluster CronJob) so new
recipes/coordinates appear automatically as the feed grows.
- TG2↔TG3 metadata-key contract test. Against a real TG2-produced
started.json
fixture, assert that every Header.extra configuration_value key TG3 configures
has a matching key emitted by TG2 (and vice versa). A drift fails CI rather than
silently rendering blank provenance columns.
Out of scope. The publisher (TG2); the API/UI (TG4a/TG4b); provisioning the host
cluster (TG1). We do not fork the worker source — only their deployment + an
AICR-native config.
Key files. New infra/testgrid/k8s/** (worker manifests, Workload Identity SA
annotations); new tools/testgrid-config-gen/ (needs the same tools/** labeler
glob TG2 adds; a narrower CODEOWNERS entry is optional); the pinned key schema in
TG6's spec. Upstream behavior reference: test-result-ingestion-reference.md §3,
§4, §9.
Tests. config-gen unit tests: given a fixture groups/ listing, assert the
generated Configuration has the right DashboardGroup/Dashboard/TestGroup nesting,
GcsPrefix includes the bucket, every TestGroup has result_source.gcs_config,
the AICR header set matches the pinned keys, and the alerting thresholds match. The
TG2↔TG3 metadata-key contract test (above). A --dry-run text-proto golden. An
integration smoke (against an emulated bucket or a staging GCS) that the workers
produce non-empty grid//tabs//summary/ from a seeded feed, and that columns
appear in chronological order for distinct seeded runs. 75% floor on
tools/testgrid-config-gen.
Acceptance. The three upstream workers run on the eidosx host cluster with
objectAdmin and resumable queues; tools/testgrid-config-gen infers CSP-first
taxonomy from groups/ and writes a valid protobuf config; the TG2↔TG3 metadata-key
contract test passes; a seeded feed rolls up into grid//tabs//summary/ with
correct column ordering.
Child of #1263 · staged from docs/design/010-aicr-testgrid.md (TG3).
Parent: #1263 · Labels:
enhancement,area/infra(auto, GKE manifestsunder
infra/testgrid/**) · Size: L · Depends on: TG1, TG2 (needs a feed toroll up and the pinned key schema to mirror)
Goal. Deploy the upstream Kubernetes TestGrid updater/tabulator/summarizer on
the
eidosxhost cluster and an AICR-native config generator that infers theCSP-first taxonomy from the bucket layout, so the raw
groups/feed is rolled upinto the processed
grid//tabs//summary/trees the API consumes — with acontract test that pins the TG2↔TG3 metadata-key seam.
Scope.
the TG1 host cluster) with
roles/storage.objectAdmin(the workers SA from TG1):updater (30m,
groups/→grid/), tabulator (15m,grid/→tabs/), summarizer(5m,
tabs/→summary/), each with--confirm,--wait,--persist-queue,--json-logs, resumablequeue/*.json. Manifests live underinfra/testgrid/k8s/(or Helm/Kustomize as the repo prefers).tools/testgrid-config-gen/(Go), owned by TG3 — it consumes the bucket layout the workers read, not the
OCI bundle TG2 reads, so it is a distinct artifact from TG2 with its own
labeler/CODEOWNERS/coverage scope. It lists
gs://<bucket>/groups/, infers theCSP-first taxonomy (DashboardGroup=
service, Dashboard=accelerator-os,TestGroup/Tab=
intent[-platform]), and writes the protobufConfigurationtogs://<bucket>/config. Honor the upstream invariants:GcsPrefixmust includethe bucket name; each
TestGroupmust setresult_source.gcs_config; configurethe AICR column-header set keyed to the exact metadata key strings TG2 emits
(the pinned schema) as the
Header.extraarray; alerting knobs(
numFailuresToAlert=1,alertStaleResultsHours=168,minAcceptableRuns=3,brokenColumnThreshold=50%) as sane defaults;DisableProwjobAnalysis: true(nopodinfo.json).
recipes/coordinates appear automatically as the feed grows.
started.jsonfixture, assert that every
Header.extraconfiguration_valuekey TG3 configureshas a matching key emitted by TG2 (and vice versa). A drift fails CI rather than
silently rendering blank provenance columns.
Out of scope. The publisher (TG2); the API/UI (TG4a/TG4b); provisioning the host
cluster (TG1). We do not fork the worker source — only their deployment + an
AICR-native config.
Key files. New
infra/testgrid/k8s/**(worker manifests, Workload Identity SAannotations); new
tools/testgrid-config-gen/(needs the sametools/**labelerglob TG2 adds; a narrower CODEOWNERS entry is optional); the pinned key schema in
TG6's spec. Upstream behavior reference:
test-result-ingestion-reference.md§3,§4, §9.
Tests. config-gen unit tests: given a fixture
groups/listing, assert thegenerated
Configurationhas the right DashboardGroup/Dashboard/TestGroup nesting,GcsPrefixincludes the bucket, every TestGroup hasresult_source.gcs_config,the AICR header set matches the pinned keys, and the alerting thresholds match. The
TG2↔TG3 metadata-key contract test (above). A
--dry-runtext-proto golden. Anintegration smoke (against an emulated bucket or a staging GCS) that the workers
produce non-empty
grid//tabs//summary/from a seeded feed, and that columnsappear in chronological order for distinct seeded runs. 75% floor on
tools/testgrid-config-gen.Acceptance. The three upstream workers run on the
eidosxhost cluster withobjectAdminand resumable queues;tools/testgrid-config-geninfers CSP-firsttaxonomy from
groups/and writes a valid protobufconfig; the TG2↔TG3 metadata-keycontract test passes; a seeded feed rolls up into
grid//tabs//summary/withcorrect column ordering.
Child of #1263 · staged from
docs/design/010-aicr-testgrid.md(TG3).