You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To test that every object has an app label or something like a support link, I would like to formulate a test like this:
suite: test app labeltemplates:
- templates/*.yamltests:
- it: object has app labelasserts:
- equal:
path: metadata.labels.appvalue: myapp
According to documentSelector this should default select "all" documents within every (*.yaml) template file.
Now the document structure already has some template surrounded by feature flags, so some files might render empty, and thus do not comprise documents.
templates/file-v1.yaml:
{{- if .Values.features.magic }}
# see file-v2.yaml
{{- else }}
---
kind: ...
{{- end }}
templates/file-v2.yaml:
{{- if .Values.features.magic }}
---
kind: ...
{{- end }}
I would expect that asserts on all documents in all templates would somehow accept empty templates, but the assert fails if not at least one document can be found per template. As I see certainly why asserts should raise if no document is fulfilling the assert, I would like to know if it is possible to test this similarly?
The unittests are indeed very strict in validating "all" templates. So the validation will fail when an empty file is part of the assertions. This setup is by design.
The best way to validate all templates is by ensure "all" template do get generated, which can be done by setting the features.magic values as part of the test. The following namesOveride test shows a similar approach where the assertions are all validated against to mention list of templates, where the ingess.enabled value has been set to true, otherwise the ingress template would result in an empty file (failing the assertion).
@mszygenda that is a great addition but too bad we can't target all documents... :(
I have same sort of assert to validate documents name but it apply to everything, that would have been great to be able to just add this:
@quintush and the solution to set specific value as part of the test don't work because for a specific value I have mutual exclusive document generation, so I always have one or another that is empty...
"description": "The path of the key to be match and the match value to assert. Using this information, helm-unittest will automatically discover the documentIndex. Generally you can ignored this field if the template file render only one document.",
"markdownDescription": "**documentSelector** (object) _optional_\n\nThe path of the key to be match and the match value to assert.\n\nUsing this information, helm-unittest will automatically discover the documentIndex.\n\nGenerally you can ignored this field if the template file render only one document.",
"properties": {
"path": {
"type": "string",
"description": "The `documentSelector` path to assert.",
"markdownDescription": "**path** (string)\n\nThe `documentSelector` path to assert."
With the restructuring of the test-job and assertion options, it is possible to use the documentSelector with only the 'skipEmptyTemplates' feature on test-job level.
The jsonschema and documentation are not yet updated, but you can check if it resolves the issue.
Activity
quintush commentedon May 10, 2024
Hello @epDHowwD,
The unittests are indeed very strict in validating "all" templates. So the validation will fail when an empty file is part of the assertions. This setup is by design.
The best way to validate all templates is by ensure "all" template do get generated, which can be done by setting the features.magic values as part of the test. The following namesOveride test shows a similar approach where the assertions are all validated against to mention list of templates, where the ingess.enabled value has been set to true, otherwise the ingress template would result in an empty file (failing the assertion).
Hopes this helps,
Greetings,
@quintush
mszygenda commentedon May 20, 2024
I believe use-case described here is manifestation of the same issue described here:
#308
And related PR I prepared to tackle it.
#335
The idea is that you could change the default behavior requiring document in each template by using documentSelector.skipEmptyTemplates property.
llavaud commentedon Mar 14, 2025
@mszygenda that is a great addition but too bad we can't target all documents... :(
I have same sort of assert to validate documents name but it apply to everything, that would have been great to be able to just add this:
@quintush and the solution to set specific value as part of the test don't work because for a specific value I have mutual exclusive document generation, so I always have one or another that is empty...
lucasfcnunes commentedon Jul 9, 2025
It seems that
documentSelector.valuecan be an object. How can a regex be made using it?helm-unittest/schema/helm-testsuite.json
Lines 1154 to 1172 in 115964f
I achieved what I needed by using https://github.com/vmware-labs/yaml-jsonpath features.
Without relying in jsonpath features the document selector is TOO STRICT #636 (comment)
quintush commentedon Aug 2, 2025
Hello @epDHowwD , @lucasfcnunes , @llavaud , @mszygenda
With the restructuring of the test-job and assertion options, it is possible to use the documentSelector with only the 'skipEmptyTemplates' feature on test-job level.
The jsonschema and documentation are not yet updated, but you can check if it resolves the issue.
Greetings
@quintush