Skip to content

Commit 868b97d

Browse files
authored
fix(kubevirt): let tenant admins upload a VMDisk image with virtctl (#5114)
## What this PR does A VMDisk created with `source: upload` waits in `UploadReady` for an image, but no tenant role can fill it. `virtctl image-upload` reads the disk PVC and CDI's PVC Prime and requests an upload token, and the tenant roles grant neither `get` on persistentvolumeclaims nor `create` on `upload.cdi.kubevirt.io/uploadtokenrequests`, so the command stops on a forbidden PVC read. `cozy:tenant:admin:base` now grants both. Tenant admins and super-admins get them in their own namespaces through the existing RoleBindings, the use and view tiers do not. Reading the DataVolume is already granted for the disk's own name by the vm-disk chart's dashboard role, so no DataVolume rule is added. The vm-disk notes print the command with `--no-create`, because the chart has already created the DataVolume, and the security self-assessment and threat model list the new grants. This is the RBAC part of #3847, split out so tenants can upload while the console upload panel and the upload proxy certificate stay in review there. The rules, their tests, the notes change and the docs lines are from @myasnikovdaniil, who is co-author on the commit. Checked live on a cluster with KubeVirt v1.8.4 and CDI v1.66.1, with virtctl v1.8.4 impersonating a user in the `<tenant>-admin` group. - With the stock roles `virtctl image-upload dv vm-disk-<name> --no-create` fails with `persistentvolumeclaims "vm-disk-<name>" is forbidden`. - With exactly these two rules added as a temporary Role, the same command reads `prime-<uid>`, uploads a 16 MiB image and the DataVolume reaches `Succeeded`. `get` on the DataVolume came from the vm-disk dashboard role. - The upload went through a port-forward to `cdi-uploadproxy` with `--insecure`. A trusted certificate on the published proxy is the other half of #3847 and is not changed here. Unit tests with `helm unittest` 1.1.2, the version CI pins. `packages/system/cozystack-basics` passes 72 of 72, against 66 on main, and `packages/apps/vm-disk` 117 of 117, against 116. Removing the upload token rule fails one case, and dropping `--no-create` from the notes fails one case. Fixes #3759 ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. Two tenant RBAC rules, their tests, the vm-disk notes and two docs files. No values key, schema, default, README or package change, so no trigger map entry applies. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(kubevirt): tenant admins can upload an image into a VMDisk created with source upload using virtctl image-upload. The tenant admin and super-admin roles get get on persistentvolumeclaims and create on upload.cdi.kubevirt.io/uploadtokenrequests, and the vm-disk notes print the command with --no-create. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Tenant administrators can now upload images into existing VM disks. * **Improvements** * The displayed upload command now explicitly targets the release namespace and avoids creating a new disk. * **Documentation** * Security guidance now clarifies the access needed for uploads and notes that persistent-volume permissions remain limited by namespace scope. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 6f8171d + 95364b8 commit 868b97d

6 files changed

Lines changed: 130 additions & 4 deletions

File tree

‎docs/security/self-assessment.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ Actors are decomposed by **trust boundary**: two components are distinct actors
5555
| GitOps / Flux delivery layer | T1 — platform control plane | Reconciles HelmReleases/Kustomizations into arbitrary privileged manifests; effectively cluster-admin. | Tenants cannot create HelmReleases (RBAC); not tenant-network-reachable. |
5656
| `cozystack-api` aggregated API server | T1 — platform control plane | Converts tenant CRs into HelmReleases; can CRUD HelmReleases, Secrets, and Cilium policies cluster-wide. Acts as the policy-enforcing gateway. | Constrains tenants via fixed chart reference, name/quota/host validation, and admission. |
5757
| System operators / controllers | T1 — platform control plane | Domain-scoped cluster privileges (provision control planes, mint certs/secrets, program CNI). | Distinct ServiceAccounts; isolated from tenants by RBAC and network policy. |
58-
| Tenant administrator (OIDC `<tenant>-admin` / `<tenant>-super-admin`) | T2 — tenant boundary | CRUD on `apps.cozystack.io` application kinds and `sdn.cozystack.io/securitygroups` in its namespace subtree; read on pods/services/ingress plus `delete` on a fixed set (pods/services/endpoints/events); no create/update of arbitrary core resources. (The ClusterRole also lists `persistentvolumes`, but that grant is inert under the namespace-scoped RoleBinding, since PVs are cluster-scoped.) `super-admin` holds the `apps.cozystack.io/*` wildcard (and can create child `Tenant`s); `admin` has an enumerated allowlist excluding some kinds (e.g. `tenants`, `monitoring`, `etcd`, `ingress`). | Namespace-scoped RBAC + admission policies + quota; no HelmRelease or cross-tenant access. |
58+
| Tenant administrator (OIDC `<tenant>-admin` / `<tenant>-super-admin`) | T2 — tenant boundary | CRUD on `apps.cozystack.io` application kinds and `sdn.cozystack.io/securitygroups` in its namespace subtree; read on pods/services/ingress plus `delete` on a fixed set (pods/services/endpoints/events); `get` on persistentvolumeclaims and `create` on `upload.cdi.kubevirt.io/uploadtokenrequests`, the pair `virtctl image-upload` needs to upload into an existing VMDisk; no create/update of arbitrary core resources. (The ClusterRole also lists `persistentvolumes`, but that grant is inert under the namespace-scoped RoleBinding, since PVs are cluster-scoped.) `super-admin` holds the `apps.cozystack.io/*` wildcard (and can create child `Tenant`s); `admin` has an enumerated allowlist excluding some kinds (e.g. `tenants`, `monitoring`, `etcd`, `ingress`). | Namespace-scoped RBAC + admission policies + quota; no HelmRelease or cross-tenant access. |
5959
| Tenant user (OIDC `<tenant>-view` / `<tenant>-use`) | T2 — tenant boundary | Read-only view plus operational verbs (VM console/vnc/portforward, start/stop/restart). Cannot create Application CRs. | Strictly dominated by tenant-admin; cannot escalate. |
6060
| Parent vs child (nested) tenant | T2 — tenant boundary | Parent ServiceAccounts bind into descendant namespaces; children do not bind upward. Quota hierarchically capped. | Ancestor→descendant trust is one-way. |
6161
| Tenant workloads / managed services | T3 — workload boundary | Run tenant-controlled code; no cluster-API rights beyond their ServiceAccount. | Namespace-scoped RBAC/identity isolate API objects; on the network, Cilium egress confines each tenant pod to its own subtree + platform system services + `world`, so unrelated tenants cannot reach each other over the pod network (enforced on egress; ingress is open to `cluster` — see Non-goals). |

‎docs/security/threat-model.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ The actors below are decomposed **by trust boundary**, not by component: two com
3939
| GitOps / Flux delivery layer | T1 | Reconciles HelmReleases/Kustomizations into arbitrary privileged manifests; effectively cluster-admin. | Tenants cannot create HelmReleases/Kustomizations (RBAC); not tenant-network-reachable. |
4040
| `cozystack-api` aggregated API server (ServiceAccount) | T1 | CRUD HelmReleases, Secrets, and CiliumNetworkPolicies cluster-wide; reads Namespaces/RBAC/quotas/nodes. Acts as the policy-enforcing gateway that converts tenant CRs to HelmReleases. | A distinct, network-exposed attack surface from Flux; constrains tenants via fixed chart ref + name/quota/host validation + admission. |
4141
| System operators / controllers | T1 | Domain-scoped cluster privileges (provision control planes, mint certs/secrets, program CNI). | Distinct ServiceAccounts, but all cluster-admin-adjacent; isolated from tenants only by RBAC and network policy. |
42-
| Tenant administrator (OIDC groups `<tenant>-admin`, `<tenant>-super-admin`) | T2 | CRUD on `apps.cozystack.io` application kinds and `sdn.cozystack.io/securitygroups` within its namespace subtree; read on pods/services/ingress plus `delete` on a fixed set (pods/services/endpoints/events); no create/update of arbitrary core resources. (The ClusterRole also lists `persistentvolumes`, but that grant is inert under the namespace-scoped RoleBinding, since PVs are cluster-scoped.) The `<tenant>-super-admin` group holds the `apps.cozystack.io/*` wildcard (and so can create child `Tenant`s); the `<tenant>-admin` group has an enumerated allowlist that excludes some kinds (e.g. `tenants`, `monitoring`, `etcd`, `ingress`, `external-dns`, `bootbox`). | Namespace-scoped RoleBindings + admission policies + quota; no access to HelmReleases, arbitrary core writes, or other tenants. |
42+
| Tenant administrator (OIDC groups `<tenant>-admin`, `<tenant>-super-admin`) | T2 | CRUD on `apps.cozystack.io` application kinds and `sdn.cozystack.io/securitygroups` within its namespace subtree; read on pods/services/ingress plus `delete` on a fixed set (pods/services/endpoints/events); `get` on persistentvolumeclaims and `create` on `upload.cdi.kubevirt.io/uploadtokenrequests`, the pair `virtctl image-upload` needs to upload into an existing VMDisk; no create/update of arbitrary core resources. (The ClusterRole also lists `persistentvolumes`, but that grant is inert under the namespace-scoped RoleBinding, since PVs are cluster-scoped.) The `<tenant>-super-admin` group holds the `apps.cozystack.io/*` wildcard (and so can create child `Tenant`s); the `<tenant>-admin` group has an enumerated allowlist that excludes some kinds (e.g. `tenants`, `monitoring`, `etcd`, `ingress`, `external-dns`, `bootbox`). | Namespace-scoped RoleBindings + admission policies + quota; no access to HelmReleases, arbitrary core writes, or other tenants. |
4343
| Tenant user (OIDC groups `<tenant>-view`, `<tenant>-use`) | T2 | Read-only view of Application CRs, pods, and services, plus operational verbs only (VM console/vnc/portforward, start/stop/restart). Cannot create Application CRs. | Strictly dominated by tenant-admin; cannot escalate to create/delete, so it is a distinct actor. |
4444
| Parent tenant vs child (nested) tenant | T2 | Parent ServiceAccounts are bound into descendant namespaces; child ServiceAccounts are not bound upward. Quota is hierarchically capped. | Ancestor→descendant only; compromising a parent compromises its children, not the reverse. |
4545
| Tenant workloads / managed services | T3 | Run tenant-controlled code; no cluster-API rights beyond their ServiceAccount. | Namespace-scoped RBAC/identity isolate their API objects. On the network, Cilium egress confines each tenant pod to its own tenant subtree, platform system services, and `world` (out-of-cluster), so unrelated tenants cannot reach each other over the pod network — enforced on the egress side (tenant-pod ingress is open to `cluster`; see Non-goals for residual caveats). |
@@ -92,7 +92,7 @@ The following are explicitly out of scope for the platform's security guarantees
9292

9393
The single most load-bearing isolation control is the combination of two facts:
9494

95-
1. The tenant ClusterRoles (`packages/system/cozystack-basics/templates/clusterroles.yaml`) grant **no** `helm.toolkit.fluxcd.io` access and **no** create/update of arbitrary core resources — only CRUD on `apps.cozystack.io` application kinds (super-admin holds the `*` wildcard; admin an enumerated allowlist) and `sdn.cozystack.io/securitygroups`, read access to pods/services/ingresses, and (for tenant-admins) `delete` on a fixed set of core objects (pods/services/endpoints/events; the `persistentvolumes` entry in the ClusterRole is inert under the namespace-scoped RoleBinding). Tenant isolation is enforced by the *absence* of privileged grants in RBAC, not by an admission policy that whitelists `apps.cozystack.io/*` writes.
95+
1. The tenant ClusterRoles (`packages/system/cozystack-basics/templates/clusterroles.yaml`) grant **no** `helm.toolkit.fluxcd.io` access and **no** create/update of arbitrary core resources — only CRUD on `apps.cozystack.io` application kinds (super-admin holds the `*` wildcard; admin an enumerated allowlist) and `sdn.cozystack.io/securitygroups`, read access to pods/services/ingresses, and (for tenant-admins) `delete` on a fixed set of core objects (pods/services/endpoints/events; the `persistentvolumes` entry in the ClusterRole is inert under the namespace-scoped RoleBinding), `get` on persistentvolumeclaims and `create` on `upload.cdi.kubevirt.io/uploadtokenrequests` (for uploading an image into an existing VMDisk with `virtctl image-upload`). Tenant isolation is enforced by the *absence* of privileged grants in RBAC, not by an admission policy that whitelists `apps.cozystack.io/*` writes.
9696
2. The Application-to-HelmRelease conversion (`pkg/registry/apps/application/rest.go`) takes the chart reference, release-name prefix, and `valuesFrom` from server-side configuration; the tenant controls only the Helm values (`app.Spec`).
9797

9898
If a future `ApplicationDefinition`, RBAC change, or API change ever let a tenant influence the chart reference, `valuesFrom`, or write HelmReleases directly, the T2→T1 isolation would collapse. Changes touching tenant RBAC or the Application REST layer should be reviewed against this invariant.
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{{- if and .Values.source (hasKey .Values.source "upload") }}
22
To upload your disk run:
33

4-
virtctl image-upload dv -n {{ .Release.Namespace }} {{ .Release.Name }} --image-path <path_to_image>
4+
virtctl image-upload dv {{ .Release.Name }} --no-create --namespace {{ .Release.Namespace }} --image-path <path_to_image>
55
{{- end }}
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
suite: VM disk upload notes
2+
templates:
3+
- templates/NOTES.txt
4+
5+
release:
6+
name: vm-disk-demo
7+
namespace: tenant-root
8+
9+
tests:
10+
- it: reuses the chart-created DataVolume for local image upload
11+
set:
12+
source:
13+
upload: {}
14+
asserts:
15+
- matchRegexRaw:
16+
pattern: 'virtctl image-upload dv vm-disk-demo --no-create --namespace tenant-root --image-path <path_to_image>'
17+
- notMatchRegexRaw:
18+
pattern: 'virtctl image-upload dv -n'

‎packages/system/cozystack-basics/templates/clusterroles.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,17 @@ metadata:
235235
labels:
236236
rbac.cozystack.io/aggregate-to-tenant-admin: "true"
237237
rules:
238+
# virtctl reads both the disk PVC and CDI's dynamically named PVC Prime.
239+
- apiGroups: [""]
240+
resources:
241+
- persistentvolumeclaims
242+
verbs:
243+
- get
244+
- apiGroups: ["upload.cdi.kubevirt.io"]
245+
resources:
246+
- uploadtokenrequests
247+
verbs:
248+
- create
238249
- apiGroups: [""]
239250
resources:
240251
- pods
Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
suite: tenant admin CDI upload RBAC
2+
3+
templates:
4+
- templates/clusterroles.yaml
5+
6+
release:
7+
name: cozystack-basics
8+
namespace: cozy-system
9+
10+
tests:
11+
- it: lets tenant admins inspect upload PVCs and create CDI upload tokens
12+
documentSelector:
13+
path: metadata.name
14+
value: cozy:tenant:admin:base
15+
asserts:
16+
- equal:
17+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-admin"]
18+
value: "true"
19+
- contains:
20+
path: rules
21+
content:
22+
apiGroups: [""]
23+
resources: ["persistentvolumeclaims"]
24+
verbs: ["get"]
25+
- contains:
26+
path: rules
27+
content:
28+
apiGroups: ["upload.cdi.kubevirt.io"]
29+
resources: ["uploadtokenrequests"]
30+
verbs: ["create"]
31+
32+
- it: keeps the admin grant out of the use, view and service-account aggregation
33+
documentSelector:
34+
path: metadata.name
35+
value: cozy:tenant:admin:base
36+
asserts:
37+
- notExists:
38+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-use"]
39+
- notExists:
40+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-view"]
41+
- notExists:
42+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant"]
43+
44+
- it: does not grant CDI upload permissions to the use tier
45+
documentSelector:
46+
path: metadata.name
47+
value: cozy:tenant:use:base
48+
asserts:
49+
- notContains:
50+
path: rules
51+
content:
52+
apiGroups: [""]
53+
resources: ["persistentvolumeclaims"]
54+
verbs: ["get"]
55+
- notContains:
56+
path: rules
57+
content:
58+
apiGroups: ["upload.cdi.kubevirt.io"]
59+
resources: ["uploadtokenrequests"]
60+
verbs: ["create"]
61+
62+
- it: aggregates upload permissions through admin into super-admin
63+
documentSelector:
64+
path: metadata.name
65+
value: cozy:tenant:admin
66+
asserts:
67+
- equal:
68+
path: metadata.labels["rbac.cozystack.io/aggregate-to-tenant-super-admin"]
69+
value: "true"
70+
- contains:
71+
path: aggregationRule.clusterRoleSelectors
72+
content:
73+
matchLabels:
74+
rbac.cozystack.io/aggregate-to-tenant-admin: "true"
75+
76+
- it: selects admin permissions for super-admin
77+
documentSelector:
78+
path: metadata.name
79+
value: cozy:tenant:super-admin
80+
asserts:
81+
- contains:
82+
path: aggregationRule.clusterRoleSelectors
83+
content:
84+
matchLabels:
85+
rbac.cozystack.io/aggregate-to-tenant-super-admin: "true"
86+
87+
- it: does not grant CDI upload token creation to the view tier
88+
documentSelector:
89+
path: metadata.name
90+
value: cozy:tenant:view:base
91+
asserts:
92+
- notContains:
93+
path: rules
94+
content:
95+
apiGroups: ["upload.cdi.kubevirt.io"]
96+
resources: ["uploadtokenrequests"]
97+
verbs: ["create"]

0 commit comments

Comments
 (0)