Skip to content

Commit 018b499

Browse files
authored
feat(monitoring): auto-route tenant logs using namespace labels (#2195)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does A tenant that runs its own monitoring stack gets a VictoriaLogs instance, but the node fluent-bit in `cozy-monitoring` sends every container log to `global.target` (`tenant-root`), so that instance stays empty. This PR routes each namespace's container logs to the VictoriaLogs named by its `namespace.cozystack.io/monitoring` label, the label the tenant chart already sets and the per-tenant vmagent already selects on. **Chart (`packages/system/monitoring-agents`).** `templates/_tenant-log-routes.tpl` looks up the namespaces and keeps those whose label is set and names a tenant other than `global.target`. For those, fluent-bit gets one `rewrite_tag` filter with a `Rule` per target (a single emitter however many tenants are routed), one `http` output per target to `vlinsert-generic.<target>.svc:9481`, and a per-target `modify Set tenant <target>`, because the re-tagged record re-enters the filter chain where the `Match *` `Add tenant` cannot overwrite the key. Unlabelled namespaces, empty labels and labels equal to `global.target` stay on the default outputs. **Controller (`internal/controller/tenantlogrouting`).** `lookup` results are not part of the digest helm-controller compares, so the chart alone would not pick up a tenant that turns monitoring on, or drop one that is removed, until an unrelated upgrade. The new reconciler in cozystack-controller digests the monitoring label across all namespaces and, when the digest moves, sets `reconcile.fluxcd.io/requestedAt` and `reconcile.fluxcd.io/forceAt` on `cozy-monitoring/monitoring-agents`, which forces an upgrade and re-runs the lookup. It digests every labelled namespace rather than repeating the chart's selection, so it cannot drift from the chart; the cost is an occasional upgrade that renders the same config. The digest it acted on is kept in its own annotation, so a manual `flux reconcile` does not make it force another upgrade. The existing ClusterRole already grants the verbs it needs. **Behaviour change for the platform operator.** Routed records are re-tagged with `keep false`: once a tenant runs its own monitoring, its namespaces' container logs go to that tenant's VictoriaLogs and no longer to the root one. Events and audit logs are not routed and stay on the root store. **Why not a per-tenant collector.** An earlier comment on this PR proposed a fluent-bit per tenant, following the per-tenant vmagent and the OTLP collector from #3763. That collector receives OTLP over the network; a log collector has to tail `/var/log` through `hostPath`, which the PodSecurity level of tenant namespaces rejects, and running it in a privileged system namespace instead would mean one DaemonSet per tenant tailing every node's logs. The routing therefore stays in the one node DaemonSet, with the reconciler closing the reconcile gap. Tests: `tests/fluent-bit_tenant_log_routing_test.yaml` mocks the namespaces with `kubernetesProvider` and pins which get a rule and an output, which are skipped (unlabelled, empty label, label equal to `global.target`), and that a routed record carries the target tenant. Three of its cases fail against the previous revision of this branch. `tests/fluent-bit_tenant_log_routing_offline_test.yaml` pins that an offline render routes nothing. `internal/controller/tenantlogrouting/reconciler_test.go` covers first sight, no-op while labels are unchanged (including after a manual reconcile), a tenant turning monitoring on, a routed namespace being deleted, and a missing release. Refs #2194 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> No entry of the trigger map matches the diff: it touches `packages/system/monitoring-agents` (values, a template helper, tests), a new reconciler in `internal/controller`, its registration in `cmd/cozystack-controller`, and a comment in the cozystack-controller ClusterRole. The one open question is `cozystack/website`: `content/en/docs/next/operations/services/monitoring/logs.md` describes log collection in general terms and does not say where a tenant's container logs land, so the behaviour change above could deserve a sentence there. I have left every box empty for a maintainer to decide. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note feat(monitoring): container logs from a tenant that runs its own monitoring stack now go to that tenant's VictoriaLogs instead of the root one. The node fluent-bit routes them by the namespace.cozystack.io/monitoring label, and cozystack-controller re-renders the node agents when that label changes on any namespace. ```
2 parents 811f030 + e510d49 commit 018b499

8 files changed

Lines changed: 546 additions & 0 deletions

File tree

‎cmd/cozystack-controller/main.go‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ import (
4646
"github.com/cozystack/cozystack/internal/controller"
4747
"github.com/cozystack/cozystack/internal/controller/cacert"
4848
"github.com/cozystack/cozystack/internal/controller/tenantgateway"
49+
"github.com/cozystack/cozystack/internal/controller/tenantlogrouting"
4950
"github.com/cozystack/cozystack/internal/controller/tenantquota"
5051
"github.com/cozystack/cozystack/internal/controller/wildcardsecret"
5152
"github.com/cozystack/cozystack/internal/telemetry"
@@ -287,6 +288,13 @@ func main() {
287288
os.Exit(1)
288289
}
289290

291+
if err = (&tenantlogrouting.Reconciler{
292+
Client: mgr.GetClient(),
293+
}).SetupWithManager(mgr); err != nil {
294+
setupLog.Error(err, "unable to create controller", "controller", "TenantLogRouting")
295+
os.Exit(1)
296+
}
297+
290298
if err = (&wildcardsecret.Reconciler{
291299
Client: mgr.GetClient(),
292300
Reader: mgr.GetAPIReader(),
Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
/*
2+
Copyright 2026 The Cozystack Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
// Package tenantlogrouting keeps the node fluent-bit's tenant log routes in
18+
// step with the namespaces that declare them.
19+
//
20+
// The monitoring-agents chart builds its routes with a Helm lookup of every
21+
// Namespace's namespace.cozystack.io/monitoring label (see
22+
// packages/system/monitoring-agents/templates/_tenant-log-routes.tpl).
23+
// helm-controller only upgrades a release when its chart or values digest
24+
// changes, and lookup results are part of neither, so a tenant that turns
25+
// monitoring on would get no route, and a removed one would keep its route,
26+
// until some unrelated change upgraded the release. This reconciler closes
27+
// that gap: it digests the label across all namespaces and, when the digest
28+
// moves, requests a forced upgrade of the release, which re-runs the lookup.
29+
//
30+
// The digest covers every namespace carrying the label, whatever its value,
31+
// rather than repeating the chart's selection (non-empty and not
32+
// global.target). A superset can only cost an upgrade that renders the same
33+
// config, while a copy of the rule could drift from the chart and miss one.
34+
// For the same reason a release that carries no digest yet is forced once on
35+
// first sight: that also covers a namespace labelled between the install
36+
// render and the first sweep.
37+
package tenantlogrouting
38+
39+
import (
40+
"context"
41+
"crypto/sha256"
42+
"encoding/hex"
43+
"sort"
44+
"strings"
45+
46+
helmv2 "github.com/fluxcd/helm-controller/api/v2"
47+
fluxmeta "github.com/fluxcd/pkg/apis/meta"
48+
corev1 "k8s.io/api/core/v1"
49+
"k8s.io/apimachinery/pkg/types"
50+
ctrl "sigs.k8s.io/controller-runtime"
51+
"sigs.k8s.io/controller-runtime/pkg/builder"
52+
"sigs.k8s.io/controller-runtime/pkg/client"
53+
"sigs.k8s.io/controller-runtime/pkg/event"
54+
"sigs.k8s.io/controller-runtime/pkg/handler"
55+
"sigs.k8s.io/controller-runtime/pkg/log"
56+
"sigs.k8s.io/controller-runtime/pkg/predicate"
57+
"sigs.k8s.io/controller-runtime/pkg/reconcile"
58+
)
59+
60+
const (
61+
// MonitoringLabel is written by the apps/tenant chart and names the tenant
62+
// whose monitoring stack a namespace reports to.
63+
MonitoringLabel = "namespace.cozystack.io/monitoring"
64+
65+
// DigestAnnotation records, on the HelmRelease, the digest the last forced
66+
// upgrade was requested for. It is compared instead of requestedAt, which
67+
// anyone running `flux reconcile` overwrites, so that a manual reconcile
68+
// does not make the next sweep force one more upgrade.
69+
DigestAnnotation = "monitoring.cozystack.io/log-routing-digest"
70+
)
71+
72+
// Release is the node-agent HelmRelease whose fluent-bit config carries the
73+
// routes (packages/core/platform/sources/monitoring-agents.yaml).
74+
var Release = types.NamespacedName{Namespace: "cozy-monitoring", Name: "monitoring-agents"}
75+
76+
// Reconciler requests a forced upgrade of Release whenever the set of
77+
// namespace monitoring labels changes.
78+
type Reconciler struct {
79+
client.Client
80+
}
81+
82+
// +kubebuilder:rbac:groups="",resources=namespaces,verbs=get;list;watch
83+
// +kubebuilder:rbac:groups=helm.toolkit.fluxcd.io,resources=helmreleases,verbs=get;list;watch;patch
84+
85+
func (r *Reconciler) Reconcile(ctx context.Context, _ ctrl.Request) (ctrl.Result, error) {
86+
logger := log.FromContext(ctx)
87+
88+
hr := &helmv2.HelmRelease{}
89+
if err := r.Get(ctx, Release, hr); err != nil {
90+
return ctrl.Result{}, client.IgnoreNotFound(err)
91+
}
92+
93+
namespaces := &corev1.NamespaceList{}
94+
if err := r.List(ctx, namespaces); err != nil {
95+
return ctrl.Result{}, err
96+
}
97+
digest := Digest(namespaces.Items)
98+
if hr.GetAnnotations()[DigestAnnotation] == digest {
99+
return ctrl.Result{}, nil
100+
}
101+
102+
patch := client.MergeFrom(hr.DeepCopy())
103+
annotations := hr.GetAnnotations()
104+
if annotations == nil {
105+
annotations = map[string]string{}
106+
}
107+
// helm-controller runs a forced upgrade when forceAt equals requestedAt and
108+
// differs from the last one it handled; the digest serves as both tokens.
109+
annotations[DigestAnnotation] = digest
110+
annotations[fluxmeta.ReconcileRequestAnnotation] = digest
111+
annotations[fluxmeta.ForceRequestAnnotation] = digest
112+
hr.SetAnnotations(annotations)
113+
if err := r.Patch(ctx, hr, patch); err != nil {
114+
return ctrl.Result{}, err
115+
}
116+
logger.Info("namespace monitoring labels changed, forcing an upgrade of the node agents", "release", Release.String(), "digest", digest)
117+
return ctrl.Result{}, nil
118+
}
119+
120+
// Digest hashes the monitoring label of every namespace that carries it, in
121+
// namespace order, so it moves exactly when a namespace gains the label,
122+
// loses it, changes its value, or is deleted.
123+
func Digest(namespaces []corev1.Namespace) string {
124+
pairs := make([]string, 0, len(namespaces))
125+
for i := range namespaces {
126+
if target, ok := namespaces[i].Labels[MonitoringLabel]; ok {
127+
pairs = append(pairs, namespaces[i].Name+"="+target)
128+
}
129+
}
130+
sort.Strings(pairs)
131+
sum := sha256.Sum256([]byte(strings.Join(pairs, "\n")))
132+
return hex.EncodeToString(sum[:8])
133+
}
134+
135+
// sweep coalesces every event into one reconcile: the digest spans all
136+
// namespaces, so there is nothing per-object to reconcile.
137+
var sweep = reconcile.Request{NamespacedName: Release}
138+
139+
func (r *Reconciler) SetupWithManager(mgr ctrl.Manager) error {
140+
toSweep := handler.EnqueueRequestsFromMapFunc(func(context.Context, client.Object) []reconcile.Request {
141+
return []reconcile.Request{sweep}
142+
})
143+
labelled := func(o client.Object) bool {
144+
_, ok := o.GetLabels()[MonitoringLabel]
145+
return ok
146+
}
147+
monitoringLabelChanged := predicate.Funcs{
148+
CreateFunc: func(e event.CreateEvent) bool { return labelled(e.Object) },
149+
DeleteFunc: func(e event.DeleteEvent) bool { return labelled(e.Object) },
150+
UpdateFunc: func(e event.UpdateEvent) bool {
151+
oldValue, oldOK := e.ObjectOld.GetLabels()[MonitoringLabel]
152+
newValue, newOK := e.ObjectNew.GetLabels()[MonitoringLabel]
153+
return oldOK != newOK || oldValue != newValue
154+
},
155+
GenericFunc: func(event.GenericEvent) bool { return false },
156+
}
157+
isRelease := predicate.NewPredicateFuncs(func(o client.Object) bool {
158+
return o.GetNamespace() == Release.Namespace && o.GetName() == Release.Name
159+
})
160+
return ctrl.NewControllerManagedBy(mgr).
161+
Named("tenantlogrouting-controller").
162+
Watches(&corev1.Namespace{}, toSweep, builder.WithPredicates(monitoringLabelChanged)).
163+
Watches(&helmv2.HelmRelease{}, toSweep, builder.WithPredicates(isRelease)).
164+
Complete(r)
165+
}
Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
/*
2+
Copyright 2026 The Cozystack Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package tenantlogrouting
18+
19+
import (
20+
"context"
21+
"testing"
22+
23+
helmv2 "github.com/fluxcd/helm-controller/api/v2"
24+
fluxmeta "github.com/fluxcd/pkg/apis/meta"
25+
corev1 "k8s.io/api/core/v1"
26+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
27+
"k8s.io/apimachinery/pkg/runtime"
28+
"sigs.k8s.io/controller-runtime/pkg/client"
29+
"sigs.k8s.io/controller-runtime/pkg/client/fake"
30+
)
31+
32+
func namespace(name string, labels map[string]string) *corev1.Namespace {
33+
return &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: name, Labels: labels}}
34+
}
35+
36+
func monitoredBy(target string) map[string]string {
37+
return map[string]string{MonitoringLabel: target}
38+
}
39+
40+
func release(annotations map[string]string) *helmv2.HelmRelease {
41+
return &helmv2.HelmRelease{ObjectMeta: metav1.ObjectMeta{
42+
Name: Release.Name,
43+
Namespace: Release.Namespace,
44+
Annotations: annotations,
45+
}}
46+
}
47+
48+
func newReconciler(t *testing.T, objs ...client.Object) *Reconciler {
49+
t.Helper()
50+
scheme := runtime.NewScheme()
51+
if err := corev1.AddToScheme(scheme); err != nil {
52+
t.Fatal(err)
53+
}
54+
if err := helmv2.AddToScheme(scheme); err != nil {
55+
t.Fatal(err)
56+
}
57+
return &Reconciler{Client: fake.NewClientBuilder().WithScheme(scheme).WithObjects(objs...).Build()}
58+
}
59+
60+
func reconcileOnce(t *testing.T, r *Reconciler) *helmv2.HelmRelease {
61+
t.Helper()
62+
if _, err := r.Reconcile(context.Background(), sweep); err != nil {
63+
t.Fatalf("Reconcile: %v", err)
64+
}
65+
hr := &helmv2.HelmRelease{}
66+
if err := r.Get(context.Background(), Release, hr); err != nil {
67+
t.Fatalf("get release: %v", err)
68+
}
69+
return hr
70+
}
71+
72+
func assertForced(t *testing.T, hr *helmv2.HelmRelease, digest string) {
73+
t.Helper()
74+
for _, key := range []string{DigestAnnotation, fluxmeta.ReconcileRequestAnnotation, fluxmeta.ForceRequestAnnotation} {
75+
if got := hr.Annotations[key]; got != digest {
76+
t.Errorf("annotation %s = %q, want %q", key, got, digest)
77+
}
78+
}
79+
}
80+
81+
func TestReconcile_ForcesUpgradeOnFirstSight(t *testing.T) {
82+
nsA := namespace("tenant-a", monitoredBy("tenant-a"))
83+
r := newReconciler(t, release(map[string]string{"cozyhr.cozystack.io/values-files": "values.yaml"}), nsA)
84+
85+
hr := reconcileOnce(t, r)
86+
87+
assertForced(t, hr, Digest([]corev1.Namespace{*nsA}))
88+
if hr.Annotations["cozyhr.cozystack.io/values-files"] != "values.yaml" {
89+
t.Errorf("unrelated annotation was dropped: %v", hr.Annotations)
90+
}
91+
}
92+
93+
func TestReconcile_LeavesReleaseAloneWhileLabelsAreUnchanged(t *testing.T) {
94+
nsA := namespace("tenant-a", monitoredBy("tenant-a"))
95+
digest := Digest([]corev1.Namespace{*nsA})
96+
// A manual `flux reconcile` has since replaced the request tokens; the
97+
// recorded digest still matches, so no further upgrade may be forced.
98+
r := newReconciler(t, release(map[string]string{
99+
DigestAnnotation: digest,
100+
fluxmeta.ReconcileRequestAnnotation: "manual",
101+
fluxmeta.ForceRequestAnnotation: "manual",
102+
}), nsA)
103+
before := reconcileOnce(t, r).ResourceVersion
104+
105+
hr := reconcileOnce(t, r)
106+
107+
if hr.ResourceVersion != before {
108+
t.Errorf("release was patched although no label changed")
109+
}
110+
if hr.Annotations[fluxmeta.ForceRequestAnnotation] != "manual" {
111+
t.Errorf("forceAt = %q, want the manual token left in place", hr.Annotations[fluxmeta.ForceRequestAnnotation])
112+
}
113+
}
114+
115+
func TestReconcile_ForcesUpgradeWhenATenantTurnsMonitoringOn(t *testing.T) {
116+
nsA := namespace("tenant-a", monitoredBy("tenant-root"))
117+
r := newReconciler(t, release(nil), nsA)
118+
stale := reconcileOnce(t, r).Annotations[DigestAnnotation]
119+
120+
nsA.Labels[MonitoringLabel] = "tenant-a"
121+
if err := r.Update(context.Background(), nsA); err != nil {
122+
t.Fatal(err)
123+
}
124+
hr := reconcileOnce(t, r)
125+
126+
if hr.Annotations[DigestAnnotation] == stale {
127+
t.Fatalf("digest did not move after tenant-a turned monitoring on")
128+
}
129+
assertForced(t, hr, Digest([]corev1.Namespace{*nsA}))
130+
}
131+
132+
func TestReconcile_ForcesUpgradeWhenARoutedNamespaceIsDeleted(t *testing.T) {
133+
nsA := namespace("tenant-a", monitoredBy("tenant-a"))
134+
nsB := namespace("tenant-b", monitoredBy("tenant-b"))
135+
r := newReconciler(t, release(nil), nsA, nsB)
136+
reconcileOnce(t, r)
137+
138+
if err := r.Delete(context.Background(), nsB); err != nil {
139+
t.Fatal(err)
140+
}
141+
hr := reconcileOnce(t, r)
142+
143+
assertForced(t, hr, Digest([]corev1.Namespace{*nsA}))
144+
}
145+
146+
func TestReconcile_NoReleaseIsNotAnError(t *testing.T) {
147+
r := newReconciler(t, namespace("tenant-a", monitoredBy("tenant-a")))
148+
if _, err := r.Reconcile(context.Background(), sweep); err != nil {
149+
t.Fatalf("Reconcile without the release: %v", err)
150+
}
151+
}
152+
153+
func TestDigest_IgnoresNamespacesWithoutTheLabel(t *testing.T) {
154+
labelled := []corev1.Namespace{*namespace("tenant-a", monitoredBy("tenant-a"))}
155+
withOthers := append([]corev1.Namespace{*namespace("cozy-system", nil)}, labelled...)
156+
if Digest(labelled) != Digest(withOthers) {
157+
t.Errorf("an unlabelled namespace moved the digest")
158+
}
159+
}
160+
161+
func TestDigest_DependsOnValueNotOrder(t *testing.T) {
162+
a := *namespace("tenant-a", monitoredBy("tenant-a"))
163+
b := *namespace("tenant-b", monitoredBy("tenant-root"))
164+
if Digest([]corev1.Namespace{a, b}) != Digest([]corev1.Namespace{b, a}) {
165+
t.Errorf("digest depends on list order")
166+
}
167+
emptied := *namespace("tenant-b", monitoredBy(""))
168+
if Digest([]corev1.Namespace{a, b}) == Digest([]corev1.Namespace{a, emptied}) {
169+
t.Errorf("emptying a label did not move the digest")
170+
}
171+
}

‎packages/system/cozystack-controller/templates/rbac.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,8 @@ rules:
2727
- apiGroups: ["cert-manager.io"]
2828
resources: ["issuers", "certificates"]
2929
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
30+
# TenantLogRoutingReconciler patches the forced-upgrade request annotations on
31+
# cozy-monitoring/monitoring-agents when a namespace monitoring label changes.
3032
- apiGroups: ["helm.toolkit.fluxcd.io"]
3133
resources: ["helmreleases"]
3234
verbs: ["get", "list", "watch", "patch", "update"]
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
{{- /*
2+
Tenant log routes for the node fluent-bit, as JSON {"<target>": ["<namespace>", ...]}.
3+
4+
The tenant chart sets namespace.cozystack.io/monitoring on every tenant
5+
namespace to the nearest tenant (itself or an ancestor) that runs its own
6+
monitoring stack, so the label names the VictoriaLogs a namespace's logs
7+
belong to. A namespace is routed when that label is set and names a tenant
8+
other than global.target; an unlabelled namespace, an empty label, or one
9+
pointing at global.target stays on the default outputs.
10+
11+
lookup results are not part of the digest helm-controller compares, so a
12+
namespace gaining, changing or losing the label does not re-render this
13+
release by itself. The tenantlogrouting reconciler in cozystack-controller
14+
(internal/controller/tenantlogrouting) forces an upgrade of this release
15+
whenever any namespace's label changes, so a new selection here that reads
16+
anything beyond that label must be taught to the reconciler too. Offline
17+
renders (helm template, dry-run) get no namespaces back from lookup and route
18+
nothing.
19+
*/ -}}
20+
{{- define "monitoring-agents.tenantLogRoutes" -}}
21+
{{- $routes := dict -}}
22+
{{- $namespaces := lookup "v1" "Namespace" "" "" -}}
23+
{{- range $ns := ($namespaces.items | default list) -}}
24+
{{- $target := index ($ns.metadata.labels | default dict) "namespace.cozystack.io/monitoring" | default "" -}}
25+
{{- if and $target (ne $target $.Values.global.target) -}}
26+
{{- $_ := set $routes $target (append (get $routes $target | default list) $ns.metadata.name) -}}
27+
{{- end -}}
28+
{{- end -}}
29+
{{- toJson $routes -}}
30+
{{- end -}}

0 commit comments

Comments
 (0)