Skip to content

Commit 9ddaefc

Browse files
authored
feat(qdrant): publish ca.crt-only trust anchor to tenants (#3341)
## What this PR does Converges qdrant onto the key-free CA trust-anchor contract (#2814), the same way #3340 does it for nats. The chart declares a `TenantProjection` sentinel naming its cert-manager CA Secret. The CA-extraction controller reads that declaration and publishes a key-free copy, `ca.crt` and nothing else, as `<release>.tenant-ca`. The qdrant ResourceDefinition selects that projection by label so it shows up in the tenant's secret list. No key-bearing Secret gains a tenant-visible label. The sentinel is gated on the chart's TLS condition. Engines whose operator mints a CA unconditionally do not need that, but qdrant only has a CA while TLS is on, and an ungated sentinel would sit `Ready=False`/`SourceNotFound` on every plaintext release. This was held while the CA-extraction controller was still open. That controller landed in #3407, so the sentinel is honoured as soon as this merges and the hold is gone. Tests pin the invariants that otherwise fail silently. The sentinel carries no `ownerReferences`, so the lineage walk falls through to the Flux release label. Neither key-bearing Certificate stamps a tenant-visible label on its Secret. The two objects that decide which Secrets a tenant may actually read, the dashboard Role with its binding and the ResourceDefinition, are pinned whole rather than checked for known-bad names: a rule granting secrets with no `resourceNames` is unrestricted rather than narrow, and a label selector can reach a key-bearing Secret while every name in the file stays correct, so neither adds a name for a probe to notice. Part of #2814. ### Follow-ups What this PR does not do, and where each belongs instead. - nats converges the same contract in #3340 and has no `dashboard-resourcemap_test.yaml`, so the dashboard Role pin belongs there too. Tracked in #3701 along with postgres, which already carries the label selector with no guard behind it. - The qdrant e2e fixture runs plaintext (`hack/e2e-chainsaw/qdrant/qdrant.yaml` sets `external: false`), so nothing in CI walks the path from the sentinel to `tenantsecrets`. postgres has a `verify-tenant-ca-projection` step to copy. - `hack/check-qdrant-rd-secrets.bats` is engine-agnostic apart from two literals, the chart path and the expected Secret set. The third copy of it is the point to parameterise over `packages/system/*-rd/cozyrds/` instead of copying again. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff, file by file. Nothing needs a manual follow-up, but not for the reason a values-only walk would give: the website reference page for an app is regenerated from that package's `README.md`, and this PR does change it. `qdrant` is in the app list hardcoded in the website `Makefile`, so the docs bot picks the new section up on the next stable tag and there is nothing to open by hand. The Terraform provider's schema is hand-written against `values.schema.json`, which is untouched, and the provider offers no typed access to the new projection, the same position it holds for every other Secret it does not model. No package is added, renamed or removed, and the `cozyrds` file is edited in place rather than renamed, so the ccp dependency-contract anchor still resolves. User-facing documentation for consuming the trust anchor ships with the chart README, matching #3340. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(qdrant): expose the CA certificate to tenants as a key-free `<release>.tenant-ca` Secret ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Tenant environments can securely access the Qdrant TLS CA certificate without receiving the private key. * TLS CA access is automatically enabled when TLS or external access is configured. * **Documentation** * Expanded TLS guidance explains certificate validation and provides commands for retrieving the tenant CA certificate. * **Tests** * Added coverage for TLS projections, secret visibility, dashboard resources, and secure fail-closed secret exposure. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 4e61c50 + 97ab512 commit 9ddaefc

7 files changed

Lines changed: 694 additions & 0 deletions

File tree

‎hack/check-qdrant-rd-secrets.bats‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
#!/usr/bin/env bats
2+
# Unit test: the qdrant ResourceDefinition hands tenants the key-free trust
3+
# anchor and nothing else.
4+
#
5+
# The chart issues qdrant-<name>-ca (CA certificate AND CA private key, the
6+
# latter under tls.key) and qdrant-<name>-tls (server certificate AND server
7+
# private key). Neither may reach a tenant: read access to the CA key would
8+
# let the holder issue certificates for anything rather than merely verify the
9+
# server. What a client actually needs is ca.crt alone, delivered by the
10+
# CA-extraction controller as the key-free <release>.tenant-ca projection and
11+
# selected by LABEL rather than by name, because a name grant conveys whatever
12+
# happens to occupy that name.
13+
#
14+
# Asserted against the RENDERED chart, not against cozyrds/qdrant.yaml on
15+
# disk. templates/cozyrd.yaml globs cozyrds/* and emits each match through
16+
# .Files.Get, so what ships is the render: reading one source file would miss
17+
# a second file in that directory, and a dotfile the glob picks up. Helm
18+
# ignores dotfiles only under templates/, so one here does ship.
19+
#
20+
# This lives in bats rather than helm-unittest because packages/system/
21+
# qdrant-rd has no `test:` target, and hack/helm-unit-tests.sh skips any
22+
# package without one. If that package ever grows one, this belongs there.
23+
#
24+
# spec.secrets is compared WHOLE rather than probed for known-bad names.
25+
# Three ways this breaks are invisible to a name-by-name check:
26+
#
27+
# - a name added under a different spelling, or via a helper;
28+
# - an include with NO resourceNames and NO matchLabels, which restricts
29+
# nothing;
30+
# - a SECOND matchLabels include selecting a label the key-bearing Secrets
31+
# also carry. cert-manager stamps controller.cert-manager.io/fao on the
32+
# Secrets it creates, both of these among them, so one such entry grants
33+
# the CA private key while every name in the file stays correct.
34+
35+
REPO_ROOT="$(cd "$(dirname "${BATS_TEST_FILENAME:-$0}")/.." && pwd)"
36+
CHART="$REPO_ROOT/packages/system/qdrant-rd"
37+
38+
# The complete tenant-visible Secret surface. The API key by name; the trust
39+
# anchor by label. Nothing else.
40+
EXPECTED_SECRETS='{"exclude":[],"include":[{"resourceNames":["qdrant-{{ .name }}-apikey"]},{"matchLabels":{"internal.cozystack.io/tenant-ca":"true"}}]}'
41+
42+
# Order within include is meaningless -- the matcher ORs across entries -- so
43+
# the array is sorted before comparing, and a missing exclude is normalized to
44+
# the empty list it means. Neither can hide a grant: sort preserves every
45+
# entry (it is not unique), and exclude only ever narrows.
46+
#
47+
# Kept as a filter string rather than a shell function because hack/cozytest.sh
48+
# inserts `return 0` before any line that is exactly `}`. A helper whose own
49+
# last command decides the outcome returns 0 regardless -- and whether a
50+
# wrapper propagates a failure at all then depends on the shell running it.
51+
# Repeating the helm invocation inline in each test costs two lines and owes
52+
# nothing to that.
53+
NORMALIZE='{exclude: (.exclude // []), include: (.include | sort)}'
54+
55+
# Every document count below uses `yq eval-all`, never `yq eval`. `eval` runs
56+
# the expression once PER DOCUMENT, so `[.] | length` reports 1 per document
57+
# rather than the number of documents -- it prints "1" for a single document,
58+
# "1\n1" for two, and "1" for empty input, which would make a count check here
59+
# pass on exactly the cases it exists to catch. `eval-all` folds the stream
60+
# into one expression. Empty input still yields 1, since yq synthesizes a null
61+
# document, so emptiness is checked separately before counting.
62+
63+
@test "qdrant-rd renders exactly one ApplicationDefinition" {
64+
# stderr is deliberately not discarded: when the render breaks, helm's own
65+
# message is the only thing that says why.
66+
rendered=$(helm template qdrant-rd "$CHART") || return 1
67+
68+
# Fail closed: an empty render makes every assertion below vacuous, and yq
69+
# cannot distinguish it from one document.
70+
if [ -z "$rendered" ]; then
71+
echo "helm template produced no output for $CHART -- guard is blind." >&2
72+
return 1
73+
fi
74+
75+
count=$(printf '%s\n' "$rendered" | yq eval-all '[.] | length' -) || return 1
76+
77+
if [ "$count" != "1" ]; then
78+
echo "Expected exactly 1 rendered document from $CHART, got: $count" >&2
79+
echo "templates/cozyrd.yaml globs cozyrds/*, so every file there -- including" >&2
80+
echo "a dotfile -- ships its own tenant grants. Extend this suite before" >&2
81+
echo "adding one." >&2
82+
return 1
83+
fi
84+
}
85+
86+
@test "qdrant-rd selects the key-free tenant-CA projection by label" {
87+
rendered=$(helm template qdrant-rd "$CHART") || return 1
88+
89+
if [ -z "$rendered" ]; then
90+
echo "helm template produced no output for $CHART -- guard is blind." >&2
91+
return 1
92+
fi
93+
94+
count=$(printf '%s\n' "$rendered" | yq eval-all \
95+
'[.. | select(has("secrets")) | .secrets.include[] | select(.matchLabels."internal.cozystack.io/tenant-ca" == "true")] | length' \
96+
-) || return 1
97+
98+
if [ "$count" -lt 1 ]; then
99+
echo 'qdrant-rd does not select internal.cozystack.io/tenant-ca: "true"' >&2
100+
echo "Without it the tenant cannot read ca.crt and TLS verification is impossible." >&2
101+
return 1
102+
fi
103+
}
104+
105+
# Whole-structure comparison. Anything added to the tenant's Secret surface --
106+
# by name, by label, or by an unrestricted entry -- changes this string.
107+
@test "qdrant-rd exposes no Secret beyond the API key and the trust anchor" {
108+
rendered=$(helm template qdrant-rd "$CHART") || return 1
109+
110+
if [ -z "$rendered" ]; then
111+
echo "helm template produced no output for $CHART -- guard is blind." >&2
112+
return 1
113+
fi
114+
115+
# eval-all again: with `eval`, a second rendered document would emit a
116+
# second JSON object and the comparison below would run on a concatenation.
117+
actual=$(printf '%s\n' "$rendered" \
118+
| yq eval-all --output-format=json '[.. | select(has("secrets")) | .secrets]' - \
119+
| jq --sort-keys --compact-output "if length == 1 then .[0] else . end | $NORMALIZE") || return 1
120+
121+
expected=$(printf '%s' "$EXPECTED_SECRETS" \
122+
| jq --sort-keys --compact-output "$NORMALIZE") || return 1
123+
124+
# Fail closed: an empty or null result means spec.secrets stopped being
125+
# readable at this path, so the guard is no longer observing what it claims.
126+
if [ -z "$actual" ] || [ "$actual" = "null" ]; then
127+
echo "spec.secrets is missing or unreadable in the rendered ResourceDefinition -- guard is blind." >&2
128+
return 1
129+
fi
130+
131+
if [ "$actual" != "$expected" ]; then
132+
echo "Unexpected tenant Secret surface in the rendered qdrant-rd spec.secrets:" >&2
133+
echo " actual: $actual" >&2
134+
echo " expected: $expected" >&2
135+
echo "qdrant-<name>-ca holds the CA PRIVATE KEY and qdrant-<name>-tls the" >&2
136+
echo "server private key. Neither may be granted to a tenant, by name or by" >&2
137+
echo "a label they also carry -- cert-manager stamps its own labels on both." >&2
138+
return 1
139+
fi
140+
}

‎packages/apps/qdrant/README.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,3 +54,17 @@ This setting is ignored if the corresponding `resources` value is set.
5454
Presets follow a cloud-style `<series>.<size>` naming convention. Five series cover the full CPU-to-memory ratio range (`t1` 1:0.5, `c1` 1:1, `s1` 1:2, `u1` 1:4, `m1` 1:8) and each series ships eight sizes (`nano` through `4xlarge`). The legacy flat names (`nano`, `micro`, `small`, `medium`, `large`, `xlarge`, `2xlarge`) remain accepted as deprecated aliases of their 1:1 instance-type equivalents.
5555

5656
See [`docs/operations/resource-presets.md`](../../../docs/operations/resource-presets.md) for the full size matrix and the legacy-to-instance-type mapping.
57+
58+
### tls
59+
60+
With TLS on, the chart issues the server certificate from a self-signed CA it creates per release. The trust anchor for verifying that certificate is published as `qdrant-<name>.tenant-ca`, where `<name>` is the name of the `Qdrant` resource: an object holding `ca.crt` and nothing else, delivered to tenants through the `core.cozystack.io/tenantsecrets` API that the base tenant roles already grant.
61+
62+
```bash
63+
kubectl --context <ctx> --namespace <tenant> \
64+
get tenantsecret qdrant-<name>.tenant-ca \
65+
--output jsonpath='{.data.ca\.crt}' | base64 --decode
66+
```
67+
68+
It is the only object that hands over the CA certificate without also handing over a private key, which is why it exists. The `qdrant-<name>-ca` Secret the chart creates stores the CA **private key** alongside the certificate — read access to it would let the holder issue certificates for anything, so it is never granted to a tenant.
69+
70+
It is reached through `tenantsecrets` rather than by reading the Secret directly, and that is deliberate: `tenantsecrets` surfaces only objects the platform has vouched for (those the application's definition selects), whereas a direct grant on the name would convey whatever happens to occupy that name.
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{{- /* Gated on the same condition as certmanager.yaml: the source Secret only exists when TLS is on. */}}
2+
{{- if eq (include "qdrant.tls.enabled" .) "true" }}
3+
---
4+
apiVersion: internal.cozystack.io/v1alpha1
5+
kind: TenantProjection
6+
metadata:
7+
name: {{ .Release.Name }}-ca
8+
spec:
9+
projections:
10+
- type: CACert
11+
sourceSecretName: {{ .Release.Name }}-ca
12+
sourceKey: ca.crt
13+
{{- end }}

‎packages/apps/qdrant/tests/certmanager_test.yaml‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,72 @@ tests:
107107
value: false
108108
documentIndex: 3
109109

110+
###################################################
111+
# Key-bearing Secrets stay invisible to tenants #
112+
###################################################
113+
114+
# Both Certificates below mint a private key. Neither may carry the labels
115+
# the tenant read path selects on; only the controller-authored projection
116+
# may.
117+
#
118+
# That path has two gates and this label is the second one. tenantsecrets
119+
# serves Secrets labelled internal.cozystack.io/tenantresource=true, and the
120+
# lineage webhook sets that by matching the Secret against the application
121+
# definition's spec.secrets selectors, one of which is the tenant-ca label
122+
# below. It only gets that far for a Secret whose owner it can resolve:
123+
# pkg/lineage walks ownerReferences, then falls back to a
124+
# helm.toolkit.fluxcd.io/name label, and returns no owner otherwise, in
125+
# which case the webhook stamps nothing at all. A Secret cert-manager writes
126+
# has neither while enableCertificateOwnerRef is false, which is how
127+
# cert-manager ships here, so the label on its own is inert today.
128+
#
129+
# It stops being inert under --enable-certificate-owner-ref, which puts an
130+
# ownerReference to the Certificate on the Secret, and the Certificate does
131+
# carry the Flux label the walk needs. Same if a secretTemplate ever copies
132+
# that label across. So the guard is not protecting against today's config;
133+
# it is keeping a cluster flag from being what decides whether the CA
134+
# private key is tenant-readable.
135+
#
136+
# Selected by the Secret each Certificate writes rather than by position: a
137+
# documentIndex would follow the slot, so reordering this template would let
138+
# a labelled key-bearing Certificate slide out from under the assertion and
139+
# pass green.
140+
- it: the key-bearing CA Secret is never marked tenant-readable
141+
release:
142+
name: test-qdrant
143+
namespace: tenant-test
144+
set:
145+
tls:
146+
enabled: true
147+
_cluster:
148+
cluster-domain: cozy.local
149+
documentSelector:
150+
path: spec.secretName
151+
value: test-qdrant-ca
152+
asserts:
153+
- notExists:
154+
path: spec.secretTemplate.labels["internal.cozystack.io/tenant-ca"]
155+
- notExists:
156+
path: spec.secretTemplate.labels["internal.cozystack.io/tenantresource"]
157+
158+
- it: the key-bearing leaf Secret is never marked tenant-readable
159+
release:
160+
name: test-qdrant
161+
namespace: tenant-test
162+
set:
163+
tls:
164+
enabled: true
165+
_cluster:
166+
cluster-domain: cozy.local
167+
documentSelector:
168+
path: spec.secretName
169+
value: test-qdrant-tls
170+
asserts:
171+
- notExists:
172+
path: spec.secretTemplate.labels["internal.cozystack.io/tenant-ca"]
173+
- notExists:
174+
path: spec.secretTemplate.labels["internal.cozystack.io/tenantresource"]
175+
110176
- it: leaf cert has server auth and client auth EKU
111177
release:
112178
name: test-qdrant

0 commit comments

Comments
 (0)