Skip to content

Commit bcc5f32

Browse files
fix(kubevirt): let tenant admins upload a VMDisk image with virtctl
A VMDisk created with source upload waits in UploadReady for an image, but no tenant role can fill it. virtctl image-upload reads the disk PVC and CDI's PVC Prime and requests an upload token, and the tenant roles grant neither get on persistentvolumeclaims nor create on uploadtokenrequests, so the command stops on a forbidden PVC read. cozy:tenant:admin:base now grants both, so tenant admins and super-admins get them in their own namespaces through the existing RoleBindings, while the use and view tiers do not. Reading the DataVolume is already granted for the disk's own name by the vm-disk chart's dashboard role. The vm-disk notes print the command with --no-create, because the chart has already created the DataVolume, and the security self-assessment and threat model list the new grants. The rules, their tests, the notes change and the docs lines come from #3847, which also carries a console upload panel and an upload proxy certificate that are still in review there. Fixes #3759 Co-authored-by: Myasnikov Daniil <[email protected]> Assisted-by: LLM Signed-off-by: Yan Bondarenko <[email protected]>
1 parent 7215b02 commit bcc5f32

6 files changed

Lines changed: 130 additions & 4 deletions

File tree

‎docs/security/self-assessment.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ Actors are decomposed by **trust boundary**: two components are distinct actors
5555
| GitOps / Flux delivery layer | T1 — platform control plane | Reconciles HelmReleases/Kustomizations into arbitrary privileged manifests; effectively cluster-admin. | Tenants cannot create HelmReleases (RBAC); not tenant-network-reachable. |
5656
| `cozystack-api` aggregated API server | T1 — platform control plane | Converts tenant CRs into HelmReleases; can CRUD HelmReleases, Secrets, and Cilium policies cluster-wide. Acts as the policy-enforcing gateway. | Constrains tenants via fixed chart reference, name/quota/host validation, and admission. |
5757
| System operators / controllers | T1 — platform control plane | Domain-scoped cluster privileges (provision control planes, mint certs/secrets, program CNI). | Distinct ServiceAccounts; isolated from tenants by RBAC and network policy. |
58-
| Tenant administrator (OIDC `<tenant>-admin` / `<tenant>-super-admin`) | T2 — tenant boundary | CRUD on `apps.cozystack.io` application kinds and `sdn.cozystack.io/securitygroups` in its namespace subtree; read on pods/services/ingress plus `delete` on a fixed set (pods/services/endpoints/events); no create/update of arbitrary core resources. (The ClusterRole also lists `persistentvolumes`, but that grant is inert under the namespace-scoped RoleBinding, since PVs are cluster-scoped.) `super-admin` holds the `apps.cozystack.io/*` wildcard (and can create child `Tenant`s); `admin` has an enumerated allowlist excluding some kinds (e.g. `tenants`, `monitoring`, `etcd`, `ingress`). | Namespace-scoped RBAC + admission policies + quota; no HelmRelease or cross-tenant access. |
58+
| Tenant administrator (OIDC `<tenant>-admin` / `<tenant>-super-admin`) | T2 — tenant boundary | CRUD on `apps.cozystack.io` application kinds and `sdn.cozystack.io/securitygroups` in its namespace subtree; read on pods/services/ingress plus `delete` on a fixed set (pods/services/endpoints/events); `get` on persistentvolumeclaims and `create` on `upload.cdi.kubevirt.io/uploadtokenrequests`, the pair `virtctl image-upload` needs to upload into an existing VMDisk; no create/update of arbitrary core resources. (The ClusterRole also lists `persistentvolumes`, but that grant is inert under the namespace-scoped RoleBinding, since PVs are cluster-scoped.) `super-admin` holds the `apps.cozystack.io/*` wildcard (and can create child `Tenant`s); `admin` has an enumerated allowlist excluding some kinds (e.g. `tenants`, `monitoring`, `etcd`, `ingress`). | Namespace-scoped RBAC + admission policies + quota; no HelmRelease or cross-tenant access. |
5959
| Tenant user (OIDC `<tenant>-view` / `<tenant>-use`) | T2 — tenant boundary | Read-only view plus operational verbs (VM console/vnc/portforward, start/stop/restart). Cannot create Application CRs. | Strictly dominated by tenant-admin; cannot escalate. |
6060
| Parent vs child (nested) tenant | T2 — tenant boundary | Parent ServiceAccounts bind into descendant namespaces; children do not bind upward. Quota hierarchically capped. | Ancestor→descendant trust is one-way. |
6161
| Tenant workloads / managed services | T3 — workload boundary | Run tenant-controlled code; no cluster-API rights beyond their ServiceAccount. | Namespace-scoped RBAC/identity isolate API objects; on the network, Cilium egress confines each tenant pod to its own subtree + platform system services + `world`, so unrelated tenants cannot reach each other over the pod network (enforced on egress; ingress is open to `cluster` — see Non-goals). |

‎docs/security/threat-model.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ The actors below are decomposed **by trust boundary**, not by component: two com
3939
| GitOps / Flux delivery layer | T1 | Reconciles HelmReleases/Kustomizations into arbitrary privileged manifests; effectively cluster-admin. | Tenants cannot create HelmReleases/Kustomizations (RBAC); not tenant-network-reachable. |
4040
| `cozystack-api` aggregated API server (ServiceAccount) | T1 | CRUD HelmReleases, Secrets, and CiliumNetworkPolicies cluster-wide; reads Namespaces/RBAC/quotas/nodes. Acts as the policy-enforcing gateway that converts tenant CRs to HelmReleases. | A distinct, network-exposed attack surface from Flux; constrains tenants via fixed chart ref + name/quota/host validation + admission. |
4141
| System operators / controllers | T1 | Domain-scoped cluster privileges (provision control planes, mint certs/secrets, program CNI). | Distinct ServiceAccounts, but all cluster-admin-adjacent; isolated from tenants only by RBAC and network policy. |
42-
| Tenant administrator (OIDC groups `<tenant>-admin`, `<tenant>-super-admin`) | T2 | CRUD on `apps.cozystack.io` application kinds and `sdn.cozystack.io/securitygroups` within its namespace subtree; read on pods/services/ingress plus `delete` on a fixed set (pods/services/endpoints/events); no create/update of arbitrary core resources. (The ClusterRole also lists `persistentvolumes`, but that grant is inert under the namespace-scoped RoleBinding, since PVs are cluster-scoped.) The `<tenant>-super-admin` group holds the `apps.cozystack.io/*` wildcard (and so can create child `Tenant`s); the `<tenant>-admin` group has an enumerated allowlist that excludes some kinds (e.g. `tenants`, `monitoring`, `etcd`, `ingress`, `external-dns`, `bootbox`). | Namespace-scoped RoleBindings + admission policies + quota; no access to HelmReleases, arbitrary core writes, or other tenants. |
42+
| Tenant administrator (OIDC groups `<tenant>-admin`, `<tenant>-super-admin`) | T2 | CRUD on `apps.cozystack.io` application kinds and `sdn.cozystack.io/securitygroups` within its namespace subtree; read on pods/services/ingress plus `delete` on a fixed set (pods/services/endpoints/events); `get` on persistentvolumeclaims and `create` on `upload.cdi.kubevirt.io/uploadtokenrequests`, the pair `virtctl image-upload` needs to upload into an existing VMDisk; no create/update of arbitrary core resources. (The ClusterRole also lists `persistentvolumes`, but that grant is inert under the namespace-scoped RoleBinding, since PVs are cluster-scoped.) The `<tenant>-super-admin` group holds the `apps.cozystack.io/*` wildcard (and so can create child `Tenant`s); the `<tenant>-admin` group has an enumerated allowlist that excludes some kinds (e.g. `tenants`, `monitoring`, `etcd`, `ingress`, `external-dns`, `bootbox`). | Namespace-scoped RoleBindings + admission policies + quota; no access to HelmReleases, arbitrary core writes, or other tenants. |
4343
| Tenant user (OIDC groups `<tenant>-view`, `<tenant>-use`) | T2 | Read-only view of Application CRs, pods, and services, plus operational verbs only (VM console/vnc/portforward, start/stop/restart). Cannot create Application CRs. | Strictly dominated by tenant-admin; cannot escalate to create/delete, so it is a distinct actor. |
4444
| Parent tenant vs child (nested) tenant | T2 | Parent ServiceAccounts are bound into descendant namespaces; child ServiceAccounts are not bound upward. Quota is hierarchically capped. | Ancestor→descendant only; compromising a parent compromises its children, not the reverse. |
4545
| Tenant workloads / managed services | T3 | Run tenant-controlled code; no cluster-API rights beyond their ServiceAccount. | Namespace-scoped RBAC/identity isolate their API objects. On the network, Cilium egress confines each tenant pod to its own tenant subtree, platform system services, and `world` (out-of-cluster), so unrelated tenants cannot reach each other over the pod network — enforced on the egress side (tenant-pod ingress is open to `cluster`; see Non-goals for residual caveats). |
@@ -92,7 +92,7 @@ The following are explicitly out of scope for the platform's security guarantees
9292

9393
The single most load-bearing isolation control is the combination of two facts:
9494

95-
1. The tenant ClusterRoles (`packages/system/cozystack-basics/templates/clusterroles.yaml`) grant **no** `helm.toolkit.fluxcd.io` access and **no** create/update of arbitrary core resources — only CRUD on `apps.cozystack.io` application kinds (super-admin holds the `*` wildcard; admin an enumerated allowlist) and `sdn.cozystack.io/securitygroups`, read access to pods/services/ingresses, and (for tenant-admins) `delete` on a fixed set of core objects (pods/services/endpoints/events; the `persistentvolumes` entry in the ClusterRole is inert under the namespace-scoped RoleBinding). Tenant isolation is enforced by the *absence* of privileged grants in RBAC, not by an admission policy that whitelists `apps.cozystack.io/*` writes.
95+
1. The tenant ClusterRoles (`packages/system/cozystack-basics/templates/clusterroles.yaml`) grant **no** `helm.toolkit.fluxcd.io` access and **no** create/update of arbitrary core resources — only CRUD on `apps.cozystack.io` application kinds (super-admin holds the `*` wildcard; admin an enumerated allowlist) and `sdn.cozystack.io/securitygroups`, read access to pods/services/ingresses, and (for tenant-admins) `delete` on a fixed set of core objects (pods/services/endpoints/events; the `persistentvolumes` entry in the ClusterRole is inert under the namespace-scoped RoleBinding), `get` on persistentvolumeclaims and `create` on `upload.cdi.kubevirt.io/uploadtokenrequests` (for uploading an image into an existing VMDisk with `virtctl image-upload`). Tenant isolation is enforced by the *absence* of privileged grants in RBAC, not by an admission policy that whitelists `apps.cozystack.io/*` writes.
9696
2. The Application-to-HelmRelease conversion (`pkg/registry/apps/application/rest.go`) takes the chart reference, release-name prefix, and `valuesFrom` from server-side configuration; the tenant controls only the Helm values (`app.Spec`).
9797

9898
If a future `ApplicationDefinition`, RBAC change, or API change ever let a tenant influence the chart reference, `valuesFrom`, or write HelmReleases directly, the T2→T1 isolation would collapse. Changes touching tenant RBAC or the Application REST layer should be reviewed against this invariant.
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{{- if and .Values.source (hasKey .Values.source "upload") }}
22
To upload your disk run:
33

4-
virtctl image-upload dv -n {{ .Release.Namespace }} {{ .Release.Name }} --image-path <path_to_image>
4+
virtctl image-upload dv {{ .Release.Name }} --no-create --namespace {{ .Release.Namespace }} --image-path <path_to_image>
55
{{- end }}
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
suite: VM disk upload notes
2+
templates:
3+
- templates/NOTES.txt
4+
5+
release:
6+
name: vm-disk-demo
7+
namespace: tenant-root
8+
9+
tests:
10+
- it: reuses the chart-created DataVolume for local image upload
11+
set:
12+
source:
13+
upload: {}
14+
asserts:
15+
- matchRegexRaw:
16+
pattern: 'virtctl image-upload dv vm-disk-demo --no-create --namespace tenant-root --image-path <path_to_image>'
17+
- notMatchRegexRaw:
18+
pattern: 'virtctl image-upload dv -n'

‎packages/system/cozystack-basics/templates/clusterroles.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,17 @@ metadata:
235235
labels:
236236
rbac.cozystack.io/aggregate-to-tenant-admin: "true"
237237
rules:
238+
# virtctl reads both the disk PVC and CDI's dynamically named PVC Prime.
239+
- apiGroups: [""]
240+
resources:
241+
- persistentvolumeclaims
242+
verbs:
243+
- get
244+
- apiGroups: ["upload.cdi.kubevirt.io"]
245+
resources:
246+
- uploadtokenrequests
247+
verbs:
248+
- create
238249
- apiGroups: [""]
239250
resources:
240251
- pods
Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
suite: tenant admin CDI upload RBAC
2+
3+
templates:
4+
- templates/clusterroles.yaml
5+
6+
release:
7+
name: cozystack-basics
8+
namespace: cozy-system
9+
10+
tests:
11+
- it: lets tenant admins inspect upload PVCs and create CDI upload tokens
12+
documentSelector:
13+
path: metadata.name
14+
value: cozy:tenant:admin:base
15+
asserts:
16+
- equal:
17+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-admin"]
18+
value: "true"
19+
- contains:
20+
path: rules
21+
content:
22+
apiGroups: [""]
23+
resources: ["persistentvolumeclaims"]
24+
verbs: ["get"]
25+
- contains:
26+
path: rules
27+
content:
28+
apiGroups: ["upload.cdi.kubevirt.io"]
29+
resources: ["uploadtokenrequests"]
30+
verbs: ["create"]
31+
32+
- it: keeps the admin grant out of the use, view and service-account aggregation
33+
documentSelector:
34+
path: metadata.name
35+
value: cozy:tenant:admin:base
36+
asserts:
37+
- notExists:
38+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-use"]
39+
- notExists:
40+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-view"]
41+
- notExists:
42+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant"]
43+
44+
- it: does not grant CDI upload permissions to the use tier
45+
documentSelector:
46+
path: metadata.name
47+
value: cozy:tenant:use:base
48+
asserts:
49+
- notContains:
50+
path: rules
51+
content:
52+
apiGroups: [""]
53+
resources: ["persistentvolumeclaims"]
54+
verbs: ["get"]
55+
- notContains:
56+
path: rules
57+
content:
58+
apiGroups: ["upload.cdi.kubevirt.io"]
59+
resources: ["uploadtokenrequests"]
60+
verbs: ["create"]
61+
62+
- it: aggregates upload permissions through admin into super-admin
63+
documentSelector:
64+
path: metadata.name
65+
value: cozy:tenant:admin
66+
asserts:
67+
- equal:
68+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-super-admin"]
69+
value: "true"
70+
- contains:
71+
path: aggregationRule.clusterRoleSelectors
72+
content:
73+
matchLabels:
74+
rbac.cozystack.io/aggregate-to-tenant-admin: "true"
75+
76+
- it: selects admin permissions for super-admin
77+
documentSelector:
78+
path: metadata.name
79+
value: cozy:tenant:super-admin
80+
asserts:
81+
- contains:
82+
path: aggregationRule.clusterRoleSelectors
83+
content:
84+
matchLabels:
85+
rbac.cozystack.io/aggregate-to-tenant-super-admin: "true"
86+
87+
- it: does not grant CDI upload token creation to the view tier
88+
documentSelector:
89+
path: metadata.name
90+
value: cozy:tenant:view:base
91+
asserts:
92+
- notContains:
93+
path: rules
94+
content:
95+
apiGroups: ["upload.cdi.kubevirt.io"]
96+
resources: ["uploadtokenrequests"]
97+
verbs: ["create"]

0 commit comments

Comments
 (0)