Repository navigation
Moving CRDs from templates/ to crds/ results in CRDs being deleted #7279
Description
Output of helm version:
version.BuildInfo{Version:"v3.0.1", GitCommit:"7c22ef9ce89e0ebeb7125ba2ebf7d421f3e82ffa", GitTreeState:"clean", GoVersion:"go1.13.4"}
Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"a4529464e4629c21224b3d52edfe0ea91b072862", GitTreeState:"clean", BuildDate:"2018-09-09T18:02:47Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:39:02Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Cloud Provider/Platform (AKS, GKE, Minikube etc.):
N/A. Tested on Kubernetes in Docker-for-Mac. Also on on-premise Kubernetes deployments.
I have a chart with CustomResourceDefinitions defined in the templates/ directory. I used the 2to3 plugin to migrate my deployments from Helm v2 to v3. I'm now trying to move the CustomResourceDefinitions from templates/ to crds/ as per the best practices.
I'm not able to perform the move without Helm 3 deleting the CustomResourceDefinitions. I think this is a bug since the documentation is quite clear that Helm does not delete CRDs (https://helm.sh/docs/topics/charts/#custom-resource-definitions-crds)
CRDs are never deleted. Deleting a CRD automatically deletes all of the CRD’s contents across all namespaces in the cluster. Consequently, Helm will not delete CRDs.
Steps to replicate
Setup and install a minimum chart.
mkdir crd-issue
mkdir crd-issue/templates
mkdir crd-issue/crds
cd crd-issue
cat > Chart.yaml <<EOF
name: crd-issue
version: 0.0.1
EOF
cat > templates/crd.yaml <<EOF
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: greetings.example.com
spec:
group: example.com
version: v1
scope: Namespaced
names:
plural: greetings
singular: greeting
kind: Greeting
EOF
helm upgrade --install crd-issue .
Verify the CRD has been installed.
kubectl get crd greetings.example.com
NAME CREATED AT
greetings.example.com 2019-12-20T11:23:05Z
Move the CustomResourceDefinition into the crds/ directory and upgrade
mv templates/crd.yaml crds
helm upgrade --install crd-issue .
Verify the CRD has been deleted
kubectl get crd greetings.example.com
No resources found.
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "greetings.example.com" not found
Fix thoughts
I'm wondering if Helm can be updated to never issue a delete for a CRD. For example by placing guards around calls to Kubernetes Delete API.
I'm happy to have a go at implementing such a feature. It'll be good to get a maintainer's thought about where best to put such a guard.
Activity
bacongobbler commentedon Dec 20, 2019
The docs may be a little misleading here. If the CRD was installed as part of the
crdsdirectory, then the CRD will not be removed onhelm uninstall. However, if it was installed as part of thetemplatesdirectory, it is treated as any other resource, and therefore it will be deleted upon removal from the chart. In other words, this is intentional behaviour.If you try installing a chart with the CRD in the
crdsdirectory first, then try removing it before upgrading, do you see the same behaviour?PhilGrayson commentedon Dec 20, 2019
I do not see the same behaviour. If the chart started with a CRD in the
crds/directory, Helm does not delete the CRD if thecrdsdirectory is removed and the release upgrade. Nor does Helm delete the CRD if the release is deleted. This is consistent with the documentation.However, for charts with CRDs in the
templates/directory (ie charts prior to Helm 3, or if the chart author did not read the documentation fully and didn't use thecrds/directory), I can't find a migration path to safety where Helm doesn't delete the CRD.The reasoning for the behaviour makes sense. I do wonder if helm could better handle this case though. For example, the linter rejecting CRDs in
template/(in the same way resources withhelm.sh/hook: crd-hookis rejected). If the other idea of skipping Deletes for CRDs is not desired, a note about this case in the documentation would be good.Do not delete templated CRDs
Do not delete templated CRDs
Do not delete templated CRDs
Do not delete templated CRDs