You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4ba9270
Browse filesBrowse the repository at this point in the historyBrowse files
docs(tenant-application-logs): scope by stream filters, allowlist picker params, bound step
Use extra_stream_filters for the application scope (the identity fields are stream fields), add each picker endpoint's own parameters to the allowlist under the forced scope, clamp step so hits and stats_query_range cannot return an unbounded number of buckets, and note that the separate Merge_Log_Key fix also protects the application scope from a forged kubernetes.labels.
Assisted-by: LLM
Signed-off-by: IvanHunters <[email protected]>
On the application root, the proxy additionally scopes to one application by its **identity labels** (the same labels as #94, `apps_cozystack_io_application_{group,kind,name}`, the lineage keys sanitized as kube-state-metrics sanitizes them), set on every line at collection time by §6:
- Because the §6 identity labels and the application endpoint ship together (Phase 2), the application scope is a plain equality match, with no pod lookup. Lines from before Phase 2 do not carry the labels; they stay reachable through the namespace endpoint, which Goals and Failure cases treat as the fallback.
115
+
- Because the §6 identity labels and the application endpoint ship together (Phase 2), the application scope is a plain equality match, with no pod lookup. The identity fields are stream fields (added to `_stream_fields` in §6), so the scope uses `extra_stream_filters`, the same mechanism as the namespace scope and what VictoriaLogs indexes on. Lines from before Phase 2 do not carry the labels; they stay reachable through the namespace endpoint, which Goals and Failure cases treat as the fallback.
116
116
-`<plural>` must be a kind of `apps.cozystack.io`, `<name>` a DNS-1123 subdomain; values are quoted. Anything else is `400`. The proxy maps `<plural>` to the title-cased lineage `Kind` by reading the ApplicationDefinitions (the same mapping as #94), in one place, and uses the mapped value in `apps_cozystack_io_application_kind`, never the raw plural.
117
-
- The upstream request is built from scratch: only allowlisted parameters (`query`, `start`, `end`, `limit` (clamped), `step`, and on `tail` only `since` or `start_offset`, both capped) and no client headers. VictoriaLogs takes the tenant from `AccountID` / `ProjectID` headers; the proxy always sets them. Client `extra_filters` / `extra_stream_filters` are dropped. `/select/tenant_ids` and every non-allowlisted path are not proxied.
118
-
-**Bounded time window.** For `query`, `hits`, `stats_query_range`, and the picker endpoints the proxy requires or derives a finite `[start, end]` and clamps the maximum span; VictoriaLogs otherwise uses the oldest stored timestamp when `start` is omitted and would scan the whole retention. Inverted ranges are rejected with `400`. The `tail` resume cap (§5) is a separate bound.
117
+
- The upstream request is built from scratch: only allowlisted parameters (`query`, `start`, `end`, `limit` (clamped), `step`, each picker endpoint's own parameters (`field` for `field_values` and `stream_field_values`, and the `facets` options), and on `tail` only `since` or `start_offset`, both capped) and no client headers. VictoriaLogs takes the tenant from `AccountID` / `ProjectID` headers; the proxy always sets them. Client `extra_filters` / `extra_stream_filters` are dropped. `/select/tenant_ids` and every non-allowlisted path are not proxied.
118
+
-**Bounded time window.** For `query`, `hits`, `stats_query_range`, and the picker endpoints the proxy requires or derives a finite `[start, end]` and clamps the maximum span; VictoriaLogs otherwise uses the oldest stored timestamp when `start` is omitted and would scan the whole retention. It also clamps `step` so that `hits` and `stats_query_range`, which return one bucket per `step`, cannot produce an unbounded number of buckets over the span (the same cap on points as #94). Inverted ranges are rejected with `400`. The `tail` resume cap (§5) is a separate bound.
119
119
- Queries using `options(ignore_global_time_filter=true)` are rejected, detected by a plain substring match on `ignore_global_time_filter` anywhere in `query`, subqueries included: it disables the time bounds the proxy relies on, and on `tail` it rescans the whole retention every tick.
120
120
- Platform sources are out of reach: kube-apiserver audit and Kubernetes events carry no `kubernetes_namespace_name` in the current pipeline, so the stream filter excludes them.
121
121
@@ -134,7 +134,7 @@ Resolve a namespace to its `VLCluster` with **exactly the rule the collector use
134
134
135
135
Fluent Bit keeps the lineage labels `apps.cozystack.io/application.{group,kind,name}` as `apps_cozystack_io_application_{group,kind,name}` (a second `nest` lift on `kubernetes_labels`, renamed before `Remove_wildcard kubernetes_`, like the three kept fields today) and adds them to `_stream_fields`. These give the application scope of §3 its labels, with the same names as #94. The lineage webhook sets the labels on pods whose owning application it resolves. This is the Phase 2 collection change.
136
136
137
-
**Related forgery fix, shipped separately.** `Merge_Log On` with no `Merge_Log_Key` lets a pod's own JSON log line add top-level keys to the record: a line such as `{"kubernetes":{"namespace_name":"<other>"}}` yields a second `kubernetes` map that the `nest` lift lifts next to the real one, so a forged `kubernetes_namespace_name` can route a line into another tenant's store that that tenant's Grafana already shows today. This is live in cozystack independent of this proposal, so it deserves its own fix: set `Merge_Log_Key` (for example `Merge_Log_Key log_processed`) so a parsed line lands under one key and cannot add top-level keys. That fix carries a breaking field-schema note: fields of JSON log lines searched at the top level today (for example `level`) become `log_processed.<field>`; in-repo charts do not read them, but any external dashboard or LogsQL query that does must migrate to `log_processed.<field>`. Phase 1 of this proposal depends on that fix having landed; a test asserts a crafted line cannot appear under another namespace.
137
+
**Related forgery fix, shipped separately.** `Merge_Log On` with no `Merge_Log_Key` lets a pod's own JSON log line add top-level keys to the record: a line such as `{"kubernetes":{"namespace_name":"<other>"}}` yields a second `kubernetes` map that the `nest` lift lifts next to the real one, so a forged `kubernetes_namespace_name` can route a line into another tenant's store that that tenant's Grafana already shows today. The same way, a line setting `kubernetes.labels` could forge another application's identity labels, so this fix also protects the application scope above. This is live in cozystack independent of this proposal, so it deserves its own fix: set `Merge_Log_Key` (for example `Merge_Log_Key log_processed`) so a parsed line lands under one key and cannot add top-level keys. That fix carries a breaking field-schema note: fields of JSON log lines searched at the top level today (for example `level`) become `log_processed.<field>`; in-repo charts do not read them, but any external dashboard or LogsQL query that does must migrate to `log_processed.<field>`. Phase 1 of this proposal depends on that fix having landed; a test asserts a crafted line cannot appear under another namespace.
0 commit comments