Skip to content

Commit 0e7f83c

Browse files
IvanHuntersIvanHunters
committed
fix(tenant): target the monitoring owner for ingest egress at any depth
Sub-tenant monitoring agents that resolve to an ancestor shipped metrics and logs through a namespace-prefix walk that never reached tenant-root from below the first level. Target the inherited monitoring owner namespace directly with one egress rule covering vminsert and vlinsert, so delivery works at any depth. Drop the -ingress-monitoring policy: the ancestor already accepts the ingress through its allow-external-communication rule (fromEntities: cluster), so the separate policy added nothing. Assisted-by: LLM Signed-off-by: IvanHunters <[email protected]>
1 parent 1a13c69 commit 0e7f83c

2 files changed

Lines changed: 92 additions & 65 deletions

File tree

‎packages/apps/tenant/templates/networkpolicy.yaml‎

Lines changed: 23 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -41,22 +41,31 @@ spec:
4141
- toEndpoints:
4242
- matchLabels:
4343
"k8s:io.cilium.k8s.namespace.labels.tenant.cozystack.io/{{ include "tenant.name" . }}": ""
44-
{{- if ne (include "tenant.name" .) "tenant-root" }}
44+
{{- /*
45+
Monitoring agents in this tenant (and the guest clusters inside it) that
46+
resolve monitoring to an ancestor ship metrics (vminsert) and logs
47+
(vlinsert) to that owner. The owner namespace is the one computed in
48+
namespace.yaml and inherited through _namespace.monitoring, so this reaches
49+
the real owner at any depth (including tenant-root), unlike a namespace
50+
prefix walk, which never reaches tenant-root from below the first level.
51+
Skipped when this tenant owns its monitoring (writes stay in-namespace) or
52+
when no ancestor does.
53+
*/}}
54+
{{- $monitoringOwner := (.Values._namespace | default dict).monitoring | default "" }}
55+
{{- if .Values.monitoring }}
56+
{{- $monitoringOwner = include "tenant.name" . }}
57+
{{- end }}
58+
{{- if and $monitoringOwner (ne $monitoringOwner (include "tenant.name" .)) }}
4559
- toEndpoints:
46-
{{- if hasPrefix "tenant-" .Release.Namespace }}
47-
{{- $parts := splitList "-" .Release.Namespace }}
48-
{{- range $i, $v := $parts }}
49-
{{- if ne $i 0 }}
50-
- matchLabels:
51-
"k8s:app.kubernetes.io/name": "vminsert"
52-
"k8s:io.kubernetes.pod.namespace": {{ join "-" (slice $parts 0 (add $i 1)) }}
5360
- matchLabels:
54-
"k8s:app.kubernetes.io/name": "vlinsert"
55-
"k8s:io.kubernetes.pod.namespace": {{ join "-" (slice $parts 0 (add $i 1)) }}
56-
{{- end }}
57-
{{- end }}
58-
{{- end }}
59-
{{- end }}
61+
"k8s:io.kubernetes.pod.namespace": {{ $monitoringOwner }}
62+
matchExpressions:
63+
- key: "k8s:app.kubernetes.io/name"
64+
operator: In
65+
values:
66+
- vminsert
67+
- vlinsert
68+
{{- end }}
6069
{{- if ne (include "tenant.name" .) "tenant-root" }}
6170
- toEndpoints:
6271
{{- if hasPrefix "tenant-" .Release.Namespace }}
@@ -146,33 +155,6 @@ spec:
146155
policy.cozystack.io/allow-to-etcd: "true"
147156
{{- end }}
148157
---
149-
{{- if .Values.monitoring }}
150-
# Sub-tenants (and the guest clusters inside them) that resolve their
151-
# monitoring to this tenant ship logs and metrics to its vlinsert / vminsert.
152-
# The {{ include "tenant.name" . }}-egress rule above opens the write hop to
153-
# the ancestor; this opens the matching ingress on the ingest endpoints,
154-
# scoped to namespaces whose namespace.cozystack.io/monitoring points here.
155-
# Without it the egress allow has no counterpart and delivery is dropped.
156-
apiVersion: cilium.io/v2
157-
kind: CiliumClusterwideNetworkPolicy
158-
metadata:
159-
name: {{ include "tenant.name" . }}-ingress-monitoring
160-
spec:
161-
endpointSelector:
162-
matchLabels:
163-
"k8s:io.kubernetes.pod.namespace": "{{ include "tenant.name" . }}"
164-
matchExpressions:
165-
- key: "k8s:app.kubernetes.io/name"
166-
operator: In
167-
values:
168-
- vlinsert
169-
- vminsert
170-
ingress:
171-
- fromEndpoints:
172-
- matchLabels:
173-
"k8s:io.cilium.k8s.namespace.labels.namespace.cozystack.io/monitoring": "{{ include "tenant.name" . }}"
174-
{{- end }}
175-
---
176158
apiVersion: cilium.io/v2
177159
kind: CiliumNetworkPolicy
178160
metadata:

‎packages/apps/tenant/tests/networkpolicy_monitoring_logs_test.yaml‎

Lines changed: 69 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,12 @@
1-
suite: tenant monitoring ingest (logs and metrics) crosses the tenant boundary
2-
# A sub-tenant (and the guest clusters inside it) that resolves its monitoring
3-
# to an ancestor must be able to write both metrics (vminsert) and logs
4-
# (vlinsert) to that ancestor's ingest endpoints. The egress opens the write
5-
# hop; the ancestor's ingress must accept it.
1+
suite: tenant monitoring ingest egress reaches the monitoring owner at any depth
2+
# A tenant (and the guest clusters inside it) that resolves its monitoring to an
3+
# ancestor must be able to write both metrics (vminsert) and logs (vlinsert) to
4+
# that owner's ingest endpoints. The owner is inherited through
5+
# _namespace.monitoring, so the egress targets the real owner directly rather
6+
# than walking the namespace prefix (which never reaches tenant-root from below
7+
# the first level). The ancestor already accepts the ingress via its
8+
# allow-external-communication policy (fromEntities: cluster), so no extra
9+
# ingress policy is rendered here.
610
templates:
711
- templates/networkpolicy.yaml
812
release:
@@ -12,55 +16,96 @@ set:
1216
_cluster:
1317
bundle-name: paas-full
1418
tests:
15-
- it: lets a sub-tenant egress to the ancestor's vlinsert as well as vminsert
19+
- it: egresses to the parent owner's vminsert and vlinsert in one rule
1620
release:
1721
name: tenant-sub
1822
namespace: tenant-ktj
1923
set:
2024
_cluster:
2125
bundle-name: paas-full
26+
_namespace:
27+
monitoring: tenant-ktj
2228
documentSelector:
2329
path: metadata.name
2430
value: tenant-ktj-sub-egress
2531
asserts:
2632
- isKind:
2733
of: CiliumClusterwideNetworkPolicy
2834
- contains:
29-
path: spec.egress[1].toEndpoints
35+
path: spec.egress
3036
content:
31-
matchLabels:
32-
"k8s:app.kubernetes.io/name": "vminsert"
33-
"k8s:io.kubernetes.pod.namespace": tenant-ktj
37+
toEndpoints:
38+
- matchLabels:
39+
"k8s:io.kubernetes.pod.namespace": tenant-ktj
40+
matchExpressions:
41+
- key: "k8s:app.kubernetes.io/name"
42+
operator: In
43+
values:
44+
- vminsert
45+
- vlinsert
46+
47+
- it: targets the owning tenant-root even when the owner is not the parent
48+
# Common layout: monitoring lives only in tenant-root and every descendant
49+
# inherits it. A namespace-prefix walk from tenant-ktj-a would emit
50+
# tenant-ktj and tenant-ktj-a and never tenant-root, dropping the traffic.
51+
release:
52+
name: tenant-b
53+
namespace: tenant-ktj-a
54+
set:
55+
_cluster:
56+
bundle-name: paas-full
57+
_namespace:
58+
monitoring: tenant-root
59+
documentSelector:
60+
path: metadata.name
61+
value: tenant-ktj-a-b-egress
62+
asserts:
63+
- isKind:
64+
of: CiliumClusterwideNetworkPolicy
3465
- contains:
35-
path: spec.egress[1].toEndpoints
66+
path: spec.egress
3667
content:
37-
matchLabels:
38-
"k8s:app.kubernetes.io/name": "vlinsert"
39-
"k8s:io.kubernetes.pod.namespace": tenant-ktj
68+
toEndpoints:
69+
- matchLabels:
70+
"k8s:io.kubernetes.pod.namespace": tenant-root
71+
matchExpressions:
72+
- key: "k8s:app.kubernetes.io/name"
73+
operator: In
74+
values:
75+
- vminsert
76+
- vlinsert
4077

41-
- it: opens the ancestor's ingress on vlinsert and vminsert for its sub-tenants
78+
- it: renders no monitoring egress rule when the tenant owns its monitoring
79+
# Owner resolves to the tenant itself, so writes stay in-namespace (covered
80+
# by allow-internal-communication) and no cross-namespace rule is added.
4281
set:
4382
monitoring: true
4483
_cluster:
4584
bundle-name: paas-full
4685
documentSelector:
4786
path: metadata.name
48-
value: tenant-ktj-ingress-monitoring
87+
value: tenant-ktj-egress
4988
asserts:
5089
- isKind:
5190
of: CiliumClusterwideNetworkPolicy
5291
- equal:
53-
path: spec.endpointSelector.matchExpressions[0].values
92+
path: spec.egress
5493
value:
55-
- vlinsert
56-
- vminsert
57-
- equal:
58-
path: spec.ingress[0].fromEndpoints[0].matchLabels["k8s:io.cilium.k8s.namespace.labels.namespace.cozystack.io/monitoring"]
59-
value: tenant-ktj
94+
- toEndpoints:
95+
- matchLabels:
96+
"k8s:io.cilium.k8s.namespace.labels.tenant.cozystack.io/tenant-ktj": ""
97+
- toEndpoints:
98+
- matchLabels:
99+
"k8s:app.kubernetes.io/instance": "etcd"
100+
"k8s:io.kubernetes.pod.namespace": tenant-root
101+
- toEndpoints:
102+
- matchLabels:
103+
cozystack.io/service: ingress
104+
"k8s:io.kubernetes.pod.namespace": tenant-root
60105

61-
- it: does not open the ingest ingress when the tenant has no monitoring
106+
- it: does not render the removed ingress-monitoring policy
62107
set:
63-
monitoring: false
108+
monitoring: true
64109
_cluster:
65110
bundle-name: paas-full
66111
asserts:

0 commit comments

Comments
 (0)