Skip to content

harden(recipes): direct dependencyRefs from monitoring-CR creators to prometheus-operator-crds#1100

Merged
mchmarny merged 1 commit into
mainfrom
fix/monitoring-cr-deps-928
May 29, 2026
Merged

harden(recipes): direct dependencyRefs from monitoring-CR creators to prometheus-operator-crds#1100
mchmarny merged 1 commit into
mainfrom
fix/monitoring-cr-deps-928

Conversation

@mchmarny

Copy link
Copy Markdown
Member

Summary

Add direct dependencyRefs: [prometheus-operator-crds] to every base component whose Helm templates emit monitoring.coreos.com/v1 CRs (nvsentinel, k8s-ephemeral-storage-metrics) so the DAG-stratified helmfile layout survives refactors of intermediate nodes, plus a Go test that enforces the invariant.

Motivation / Context

After #926 split prometheus-operator-crds out as its own release, several components that create monitoring.coreos.com/v1 CRs (PodMonitor / ServiceMonitor / etc.) reached the CRDs only via a multi-hop transitive path:

nvsentinel → gpu-operator → kube-prometheus-stack → prometheus-operator-crds

If anyone removes the gpu-operator → kube-prometheus-stack edge in recipes/overlays/base.yaml — plausible since gpu-operator doesn't strictly need kube-prometheus-stack to install — the helm-diff race on nvsentinel's PodMonitor CR silently returns. No CI test catches it; the failure surfaces only on a fresh-cluster helmfile apply. This PR closes that gap by making the edge direct and asserting the invariant in a unit test.

Fixes: #928
Related: #914, #926

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Component(s) Affected

  • Recipe engine / data (pkg/recipe)

Implementation Notes

Audit method. Rendered each Helm chart in recipes/overlays/base.yaml (and registry/mixins) at its pinned version with its checked-in values file, then grepped for apiVersion: monitoring.coreos.com. Install-time emitters at our values:

Component CR kinds emitted Direct edge before After
kube-prometheus-stack Alertmanager, Prometheus, PrometheusRule, ServiceMonitor (50) ✅ (unchanged)
nvsentinel PodMonitor (1) ❌ (transitive)
k8s-ephemeral-storage-metrics ServiceMonitor (1) ❌ (transitive)

Other charts whose RBAC apiGroups mention monitoring.coreos.com (gpu-operator, network-operator, k8s-nim-operator, kai-scheduler, slinky-slurm-operator) do not emit any monitoring CR in their install-time templates — the operators create those CRs at runtime once their own CRD is reconciled, which is after install ordering matters.

Test. TestMonitoringCREmittersHaveDirectPrometheusOperatorCRDsEdge in pkg/recipe/yaml_test.go enforces a direct edge (not just closure membership). Closure is too weak — the very failure the issue describes is "transitive only" silently breaking. The test body documents the helm template … | grep -c apiVersion: monitoring.coreos.com refresh recipe for chart bumps.

nvsentinel registry comment. Reworded to reflect the direct edge in base.yaml instead of the four-hop chain.

Testing

unset GITLAB_TOKEN && make qualify   # passes: tests + race + lint + e2e (22/22 chainsaw) + scan
make bom-docs                        # no diff (no chart pins / values changed)

Per-package coverage on pkg/recipe:

  • Baseline (origin/main): 91.3%
  • This branch: 91.3% (test-only addition)

Risk Assessment

  • Low — Adds extra dependencyRefs edges to two components in base.yaml; DAG-flattening makes the extra edges no-ops while the transitive paths still exist, so install order is unchanged. The new test pins the invariant going forward.

Rollout notes: No CLI/API surface change. Generated bundles include the same components in the same install levels (verified by chainsaw cli-bundle-helmfile and cli-recipe-overlays).

Checklist

  • Tests pass locally (make test with -race)
  • Linter passes (make lint)
  • I did not skip/disable tests to make CI green
  • I added/updated tests for new functionality
  • I updated docs if user-facing behavior changed (N/A — no user-visible change)
  • Changes follow existing patterns in the codebase
  • Commits are cryptographically signed (git commit -S)

… prometheus-operator-crds

Components whose Helm templates emit monitoring.coreos.com/v1 CRs
previously relied on a multi-hop transitive path to reach
prometheus-operator-crds. Removing any intermediate edge (e.g.,
gpu-operator -> kube-prometheus-stack) would silently break the
helm-diff CRD-registration race that #914/#926 fixed.

This hardens the DAG so the direct edge survives refactors:

- nvsentinel: add prometheus-operator-crds direct dependencyRef
  (chart emits PodMonitor).
- k8s-ephemeral-storage-metrics: add prometheus-operator-crds direct
  dependencyRef (chart emits ServiceMonitor).
- nvsentinel registry comment: drop "transitively satisfied" wording.
- pkg/recipe: new TestMonitoringCREmittersHaveDirectPrometheusOperatorCRDsEdge
  fails if any known emitter (kube-prometheus-stack, nvsentinel,
  k8s-ephemeral-storage-metrics) drops the direct edge.

The emitter list is grounded in `helm template` output of each chart
at the version pinned in base.yaml using the checked-in values file;
the test body documents the refresh recipe.

Fixes: #928
Related: #914, #926
@mchmarny mchmarny requested review from a team as code owners May 29, 2026 14:26
@mchmarny mchmarny self-assigned this May 29, 2026
@mchmarny mchmarny requested a review from lalitadithya May 29, 2026 14:26
@mchmarny mchmarny enabled auto-merge (squash) May 29, 2026 14:29
@coderabbitai

coderabbitai Bot commented May 29, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 82fa816a-96af-4946-9b11-fc57acbe846d

📥 Commits

Reviewing files that changed from the base of the PR and between 8d4de99 and ceeaeb9.

📒 Files selected for processing (3)
  • pkg/recipe/yaml_test.go
  • recipes/overlays/base.yaml
  • recipes/registry.yaml

📝 Walkthrough

Walkthrough

This PR hardens the recipe dependency graph by enforcing direct dependencyRefs from monitoring CR emitter components to prometheus-operator-crds. A new validation test verifies that components emitting monitoring.coreos.com/v1 CRs declare the CRD as a direct dependency. The overlay adds this direct dependency to nvsentinel and k8s-ephemeral-storage-metrics, and the registry documentation for nvsentinel is updated to reflect the direct dependency model instead of transitive ordering.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adding direct dependencyRefs from monitoring CR emitter components to prometheus-operator-crds.
Description check ✅ Passed The description provides clear context on the motivation, implementation details, testing results, and risk assessment related to the changeset.
Linked Issues check ✅ Passed The PR fully addresses all acceptance criteria from issue #928: audited charts for monitoring CR emitters, added direct dependencyRefs for nvsentinel and k8s-ephemeral-storage-metrics, updated nvsentinel registry comment, and implemented a Go unit test enforcing the direct edge invariant.
Out of Scope Changes check ✅ Passed All changes are directly scoped to issue #928: modifications to recipes/overlays/base.yaml, recipes/registry.yaml, and the new test in pkg/recipe/yaml_test.go align with audit findings and acceptance criteria.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/monitoring-cr-deps-928

Comment @coderabbitai help to get the list of available commands and usage tips.

@mchmarny mchmarny merged commit 8396a5c into main May 29, 2026
168 of 169 checks passed
@mchmarny mchmarny deleted the fix/monitoring-cr-deps-928 branch May 29, 2026 14:39
@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report ✅

Metric Value
Coverage 77.0%
Threshold 75%
Status Pass
Coverage Badge
![Coverage](https://img.shields.io/badge/coverage-77.0%25-green)

No Go source files changed in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

harden(recipes): direct dependencyRefs from monitoring-CR creators to prometheus-operator-crds

2 participants