You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2f1bed8
Browse filesBrowse the repository at this point in the historyBrowse files
docs(tenant-resource-metrics-api): fold the approving-review follow-ups
Correct the tenant Grafana network path (the egress allowlist does not admit the proxy's namespace, so add an allow rule or run it in an allow-listed one), guard the per-application boundary against a forged identity label (the scrape relabel must win over honorLabels) and test it, note the slash as well as the dots in the sanitized label keys, drop lookback from the out-of-bounds case now that the lookup window is fixed, and add an e2e test pinning the label-endpoint filter behaviour against the shipped vmselect.
Assisted-by: LLM
Signed-off-by: IvanHunters <[email protected]>
Copy file name to clipboardExpand all lines: design-proposals/tenant-resource-metrics-api/README.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -114,7 +114,7 @@ The proxy serves two Kubernetes-shaped roots, with the native Prometheus API (`a
114
114
115
115
On the application root, the proxy additionally scopes the query to one application. The scope is defined by **what it returns**: the application's series within the namespace.
116
116
117
-
-**Where series carry the application's identity labels, it matches on them.** The labels are `apps_cozystack_io_application_{group,kind,name}` (the lineage keys with dots replaced by underscores), written directly on the series by the `scrapeClass` relabel; this is the core mechanism, not an add-on. Exporter series inside an application's pods get them by a scrape-time relabel from the pod's lineage labels (a `scrapeClass` on the cluster-wide and tenant `VMAgent`s). Note this is the series-label form; the pod/PVC lookup below instead reads kube-state-metrics' `kube_pod_labels`, where the same keys appear in KSM's prefixed `label_apps_cozystack_io_application_*` form. Node-level series such as cAdvisor and kube-state-metrics do not carry them yet; platform-side enrichment for those is part of the broader work.
117
+
- **Where series carry the application's identity labels, it matches on them.** The labels are `apps_cozystack_io_application_{group,kind,name}` (the lineage keys with the slash and the dots replaced by underscores), written directly on the series by the `scrapeClass` relabel; this is the core mechanism, not an add-on. Exporter series inside an application's pods get them by a scrape-time relabel from the pod's lineage labels (a `scrapeClass` on the cluster-wide and tenant `VMAgent`s). The relabel must win over any exporter-supplied label: application scrapes must not set `honorLabels: true` (today only the ingress module does), or the scrape class must drop an exporter-supplied `apps_cozystack_io_application_*` before setting its own, so a pod cannot forge another application's identity. Note this is the series-label form; the pod/PVC lookup below instead reads kube-state-metrics' `kube_pod_labels`, where the same keys appear in KSM's prefixed `label_apps_cozystack_io_application_*` form. Node-level series such as cAdvisor and kube-state-metrics do not carry them yet; platform-side enrichment for those is part of the broader work.
118
118
-**Until a series carries the labels, a pod/PVC lookup covers it, and becomes a fallback once it does.** The proxy resolves the application's pods and PVCs over the request window from `kube_pod_labels` / `kube_persistentvolumeclaim_labels` (via the label-values API, so pods gone before `end` are included) and ORs a `pod=~"^(<pods>)$"` branch and a `persistentvolumeclaim=~"^(<pvcs>)$"` branch, each also carrying `namespace="<ns>"`. An empty list omits its branch (never `pod=~"^()$"`, which would match series without a `pod` label). Lists are capped; above the cap the proxy answers `422` and asks for a shorter range.
119
119
-**Kind versus plural.** Lineage labels carry the title-cased kind (`Postgres`); paths and RBAC use the plural (`postgreses`). The proxy maps one to the other through ApplicationDefinitions, in one place, reused by the label match and the lookup.
120
120
-**No kind-specific branches.** A VM's host-side CPU, memory, and pod-network usage is covered by the pod branch through its `virt-launcher` pod; guest-view `kubevirt_vmi_*` series join the scope once they carry the identity labels. The proxy holds no per-kind knowledge.
@@ -138,7 +138,7 @@ The forced namespace matcher keeps reads safe, but a `namespace` label in the sh
138
138
139
139
### 8. Deployment
140
140
141
-
One small stateless Go service plus a Service, running in a platform namespace (`cozy-monitoring`). The dashboard's nginx routes the two roots to it ahead of `/apis`; a tenant's Grafanareaches it through the tenant namespace's allow-listed egress to`cozy-system`/platform services, as the dashboard already does for `vlselect`. No change to `cozystack-api`, and no new Kubernetes API surface.
141
+
One small stateless Go service plus a Service. The dashboard's nginx (in `cozy-dashboard`) routes the two roots to it ahead of `/apis`, so the SPA needs no change to tenant egress. A tenant's Grafana, however, reaches it only if network policy allows it: the tenant egress allowlist (`packages/apps/tenant/templates/networkpolicy.yaml`) today admits `cozy-dashboard`, `cozy-linstor`, `cozy-keycloak`,`cozy-grafana-operator`, `cozy-kubevirt-cdi`, and `cozy-kubevirt`, but not the namespace the proxy runs in, so either add an allow rule for the proxy shaped like `allow-to-dashboard`, or run the proxy in a namespace that is already allow-listed. No change to `cozystack-api`, and no new Kubernetes API surface.
142
142
143
143
## User-facing changes
144
144
@@ -156,7 +156,7 @@ One small stateless Go service plus a Service, running in a platform namespace (
156
156
157
157
-**Identity and policy are Kubernetes.**`TokenReview` and `SubjectAccessReview` on `apps.cozystack.io`/`cozystack.io`; the proxy stores no credentials of its own for callers and holds only its own ServiceAccount to reach `vmselect`.
158
158
-**Forced namespace matcher** is the baseline isolation hinge: injected server-side into every selector and `match[]`, with the upstream request rebuilt from an allowlist (URL and POST form) so a caller cannot smuggle `extra_filters` / `extra_filters[]` / `extra_label` to widen scope.
159
-
-**The application endpoint is a boundary for a narrowed grant.** A user whose grant is limited (via `resourceNames`) to one application cannot read another application's series in the same namespace, through `query`, `series`, or the label endpoints. A test asserts this.
159
+
-**The application endpoint is a boundary for a narrowed grant.** A user whose grant is limited (via `resourceNames`) to one application cannot read another application's series in the same namespace, through `query`, `series`, or the label endpoints. This rests on the scrape-time relabel winning over exporter-supplied labels (the `honorLabels` note in 4a), so a pod cannot forge another application's identity label. A test asserts both: the narrowed grant cannot reach another application, and an exporter emitting another application's identity label stays out of that application's scope.
160
160
-**Input.**`<plural>` must be a kind of `apps.cozystack.io`, `<name>` a DNS-1123 subdomain; all values entering filters are quoted and regex-escaped.
161
161
-**Write-path trust** is a documented weakness of the shared store (Design 6): a tenant can write another namespace's label. It cannot leak data but can fabricate or inflate another tenant's series; the fix is a write-path control chosen in the broader work, and exposure timing is a rollout decision.
162
162
-**Why not `services/proxy` to `vmselect`:** coarse authz, no forced matcher; it reads every tenant's series on the shared store.
@@ -169,7 +169,7 @@ One small stateless Go service plus a Service, running in a platform namespace (
169
169
- Caller lacks the permission for the requested scope: 403 before any query runs.
170
170
- Caller tries to drop or widen the namespace matcher (`extra_filters`, `extra_filters[]`, `extra_label`, extra `match[]`): impossible; the request is rebuilt from an allowlist with the matcher injected.
171
171
- Application scope with no pods or PVCs: the empty branch is not sent, so it returns only the application's own labelled series, never the whole namespace.
172
-
- Range, step, or lookback out of bounds: rejected by the cost limits (`422`).
172
+
- Rangeor step out of bounds: rejected by the cost limits (`422`).
173
173
- Token invalid or expired: `TokenReview` fails, 401.
174
174
175
175
## Testing
@@ -178,7 +178,7 @@ One small stateless Go service plus a Service, running in a platform namespace (
178
178
-**Per-application boundary:** a user holding only a grant for one application (via `resourceNames`) cannot read another application's series in the same namespace through `query`, `series`, or the label endpoints.
179
179
-**Application scope:** an application without PVCs (or without pods) does not see neighbour series (the empty branch is not sent); a `<name>` such as `a"} or {namespace=~".+` is rejected with `400`; an expression on a restarted application returns the series of its earlier pods within the lookback window.
180
180
-**Integration:**`TokenReview` + `SubjectAccessReview` allow `<tenant>-view` in the tenant namespace and deny a foreign tenant; a POST query is authorized the same as a GET.
181
-
-**e2e:** two tenants against the shared `tenant-root` store; tenant A reads only A's series; a VM/database graph renders end to end in the dashboard; a tenant's Grafana datasource against the proxy returns only that tenant's series.
181
+
-**e2e:** two tenants against the shared `tenant-root` store; tenant A reads only A's series; a VM/database graph renders end to end in the dashboard; a tenant's Grafana datasource against the proxy returns only that tenant's series. An e2e test also pins the label-endpoint filter behaviour against the shipped `vmselect` version (it applies the injected filters on the label endpoints while `match[]` is present, which the `-search.ignoreExtraFiltersAtLabelsAPI` help text appears to contradict), so a version bump cannot widen label reads silently.
0 commit comments