Skip to content

About

BDD styled unit test framework for Kubernetes Helm charts as a Helm plugin.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1.4k stars

Watchers

9 watching

Forks

Latest commit

 

History

1,774 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

helm unittest

Release Github Actions License Quality Gate Status OpenSSF Best Practices OpenSSF

Unit test for helm chart in YAML to keep your chart consistent and robust!

Features:

Documentation

If you are ready for writing tests, check the DOCUMENT for the test API in YAML.

Install

When not defining any versions, it will install the latest version of binary into helm plugin directory, otherwise it will install the specified version.

Using Helm 3:

helm plugin install https://github.com/helm-unittest/helm-unittest.git

Using Helm 4*:

helm plugin install https://github.com/helm-unittest/helm-unittest.git --verify=false

Using OCI download**:

helm plugin install oci://ghcr.io/helm-unittest/helm-unittest/unittest:latest

or using http download***:

helm plugin install https://github.com/helm-unittest/helm-unittest/releases/download/v${plugin_version}/unittest-${plugin_version}.tgz

Notes:
* for Helm 4, installation using webhooks GPG verification is not supported, so --verify=false is required when installing from git repository.

** when using oci download, please note the following limitations:

  • the download contains all os and architecture binaries, making the package larger;
  • the download is only supported since plugin version 1.1.0 and later;
  • for helm 4 the archive download can perform a GPG verification, when the public-key.asc is imported into the gpg store.

*** when using http download, please note the same limitations as the oci download, including:

  • the download can only have a fixed version, which needs to be filled twice in the url;
  • the archive download does not support auto update of the plugin

Importing the public key for GPG verification:

# Import the public key
curl -SsL https://github.com/helm-unittest/helm-unittest/raw/refs/heads/main/public-key.asc | gpg --import

# Convert your keyring to the legacy gpg format
# See https://helm.sh/docs/topics/provenance/
gpg --export > ~/.gnupg/pubring.gpg

Docker Usage

# run help of latest helm with latest helm unittest plugin
docker run -ti --rm -v $(pwd):/apps helmunittest/helm-unittest

# run help of specific helm version with specific helm unittest plugin version
docker run -ti --rm -v $(pwd):/apps helmunittest/helm-unittest:3.11.1-0.3.0

# run unittests of a helm 3 chart
# make sure to mount local folder to /apps in container
docker run -ti --rm -v $(pwd):/apps helmunittest/helm-unittest:3.11.1-0.3.0 .

# run unittests of a helm 3 chart with Junit output for CI validation
# make sure to mount local folder to /apps in container
# the test-output.xml will be available in the local folder.
docker run -ti --rm -v $(pwd):/apps helmunittest/helm-unittest:3.11.1-0.3.0 -o test-output.xml -t junit .

The docker container contains the fully installed helm client, including the helm-unittest plugin.

Get Started

Add tests in .helmignore of your chart, and create the following test file at $YOUR_CHART/tests/deployment_test.yaml:

suite: test deployment
templates:
  - deployment.yaml
tests:
  - it: should work
    set:
      image.tag: latest
    asserts:
      - isKind:
          of: Deployment
      - matchRegex:
          path: metadata.name
          pattern: -my-chart$
      - equal:
          path: spec.template.spec.containers[0].image
          value: nginx:latest

and run:

helm unittest $YOUR_CHART

Now there is your first test! ;)

Test Suite File

The test suite file is written in pure YAML, and default placed under the tests/ directory of the chart with suffix _test.yaml. You can also have your own suite files arrangement with -f, --file option of cli set as the glob patterns of test suite files related to chart directory, like:

helm unittest -f 'my-tests/*.yaml' -f 'more-tests/**/*.yaml' my-chart

Check DOCUMENT for more details about writing tests.

Templated Test Suites

You may find yourself needing to set up a lot of tests that are a parameterization of a single test. For instance, let's say that you deploy to 3 environments env = dev | staging | prod.

In order to do this, you can actually write your tests as a helm chart as well. If you go this route, you must set the --chart-tests-path option. Once you have done so, helm unittest will run a standard helm render against the values.yaml in your specified directory.

/my-chart
  /tests-chart
    /Chart.yaml
    /values.yaml
    /templates
      /per_env_snapshots.yaml

  /Chart.yaml
  /values.yaml
  /.helmignore
  /templates
    /actual_template.yaml

In the above example file structure, you would maintain a helm chart that will render out against the Chart.yaml that is provided and the values.yaml. With rendered charts, any test suite that is generated is automatically ran we do not look for a file suffix or glob.

Note: since you can create multiple suites in a single template file, you must provide the suite name, since we can no longer use the test suite file name meaningfully.

Note 2: since you can be running against subcharts and multiple charts, you need to make sure that you do not designate your --chart-tests-path to be the same folder as your other tests. This is because we will try to render those non-helm test folders and fail during the unit test.

Note 3: for snapshot tests, you will need to provide a helm ignore that ignores */__snapshot__/*. Otherwise, subsequent runs will try to render those snapshots.

The command for the above chart and test configuration would be:

helm unittest --chart-tests-path tests-chart my-chart

Usage

helm unittest [flags] CHART [...]

This renders your charts locally (without tiller) and runs tests defined in test suite files.

Flags

      --color                   enforce printing colored output even stdout is not a tty. Set to false to disable color
      --strict                  strict parse the testsuites (default false)
  -d  --debugPlugin             enable debug logging (default false)
  -v, --values stringArray      absolute or glob paths of values files location to override helmchart values
  -f, --file stringArray        glob paths of test files location, default to tests\*_test.yaml (default [tests\*_test.yaml])
  -q, --failfast                directly quit testing, when a test is failed (default false)
  -h, --help                    help for unittest
  -t, --output-type string      the file format in which test results are written, accepted types are (JUnit, NUnit, XUnit) (default XUnit)
  -o, --output-file string      the file where test results are written in the specified format, defaults no output is written to file
  -u, --update-snapshot         update the snapshot cached if needed, make sure you review the changes before updating
  -s, --with-subchart charts    include tests of the subcharts within charts folder (default true)
      --chart-tests-path string the folder location relative to the chart where a helm chart to render test suites is located
      --skip-schema-validation  skip values schema validation when rendering the chart (default false)
      --parallel                run test suites in parallel, ignored when --debugPlugin is set (default false)
      --max-workers int         maximum number of parallel workers, 0 means the number of CPU cores (only used with --parallel) (default 0)
      --parallel-level string   granularity of --parallel: 'suite' runs test suites concurrently, 'test' runs the test cases of all suites in one shared worker pool (only used with --parallel) (default suite)

By default test suites run sequentially, exactly as in previous releases. Pass --parallel to opt in to concurrent execution:

$ helm unittest --parallel my-chart

With --parallel, test suites run concurrently. Suites that share a snapshot (.snap) file are grouped and run sequentially within their group, so snapshot files are never written concurrently. Output stays deterministic: suites are printed in the same order as a sequential run. Combined with --failfast, the first failing suite stops any not-yet-started suites from being scheduled while in-flight suites finish. --parallel is ignored when --debugPlugin is set and execution falls back to sequential.

By default parallel execution uses one worker per CPU core. Use --max-workers to cap the number of suite groups that run at once, for example to limit resource usage in CI:

$ helm unittest --parallel --max-workers 4 ./charts/my-app

--max-workers has no effect unless --parallel is also set, and a value of 0 (the default) means one worker per CPU core.

By default --parallel runs whole test suites concurrently. When a chart has a few large suites, or many suites sharing one snapshot file, use --parallel-level test to run individual test cases concurrently instead:

$ helm unittest --parallel --parallel-level test ./charts/my-app

At the test level the test cases of all suites share one pool of --max-workers workers, so no more than that many test cases render at once. Output, counts and snapshot files are identical to a sequential run: results are reported in the order the tests are defined, and snapshots are written once per snapshot file after all of its suites finished. With --failfast, test cases after the first failing one are not reported or counted, even if they already ran, and no snapshot file is written for test cases that are not reported. Coverage is not buffered per test case, so with --coverage and --failfast the coverage table may include test cases that ran but were not reported. --parallel-level has no effect without --parallel, and --debugPlugin still forces sequential execution.

Code Coverage

Pass --coverage to print a per-template coverage table after the test summary. Each chart template is parsed via text/template/parse and instrumented with a covprobe <idx> call at every action, branch and range node; covprobe records the hit and emits nothing, so a helper whose output is consumed as data (e.g. include "..." | fromJson) still gets valid input. Each test job then renders a second time through the instrumented chart, against its own deep copy so coverage from one job can never leak into another. This coverage render is isolated from the assertion render, so adding --coverage never changes test results.

Output columns:

  • Actions — {{ ... }} expressions executed at least once.
  • Branches — if, else, with, with-else bodies.
  • Loops — range body / range-else entered at least once. The trailing "N iters" annotation is the total iteration count summed across every test run, so a range over a large list shows real loop volume even when the binary coverage stat is 100%.
  • Used — yes if the template rendered non-empty content in any test, or (for _*.tpl partials) if any of its define-block probes were hit via template / include. no flags dead templates that no test exercises, which the action/branch/loop columns alone can't always surface (e.g. a static-YAML template that's never rendered shows - for all three but no for Used). The footer row shows the rendered count as used/total.

Coverage respects the existing --with-subchart flag: when it's set to false, subchart templates are excluded from coverage instrumentation and the report. They are still copied into the chart that's rendered for coverage so include "subchart.foo" calls from the parent chart still work, only their probes and report rows are dropped.

Use --coverage-file path/to/report to emit a machine-readable report; this flag implies --coverage. The format is controlled by --coverage-format, which accepts a single format or a comma-separated list, and is validated before any chart is rendered so a typo fails the run immediately instead of producing no report file after a full test run:

Format Flag value Consumed by
Cobertura XML (default) cobertura Codecov, SonarQube, GitLab, Jenkins (Cobertura plugin), Azure DevOps
LCOV lcov Coveralls, Codecov, VS Code "Coverage Gutters", JetBrains import

Action probes are reported as line coverage (Cobertura <line> / LCOV DA); branch and loop probes are reported as branches (Cobertura condition-coverage attribute / LCOV BRDA). Templates that failed to parse appear in the report with empty counters so consumers still see them in the file list. Report paths (Cobertura filename, LCOV SF:) are rooted at the chart's actual directory on disk, not its declared Chart.yaml name, so they resolve correctly for tools that read them relative to the working directory.

A multi-chart run (helm unittest chart-a chart-b) writes one merged report per format; each chart's files are already unique (rooted at their own directory), so nothing collides.

For a single format, --coverage-file is the exact output path. For multiple formats, --coverage-file is treated as a path stem; each format appends its conventional extension automatically — and a trailing known extension on the stem (e.g. coverage.xml) is stripped first so you don't end up with coverage.xml.xml. Example:

helm unittest \
  --coverage-format cobertura,lcov \
  --coverage-file ./reports/cov \
  <chart>

# writes:
#   ./reports/cov.xml    (cobertura)
#   ./reports/cov.info   (lcov)

Yaml JsonPath Support

Now JsonPath is supported for mappings and arrays. This makes it possible to find items in an array, based on JsonPath. For more detail on the jsonPath syntax.

Due to the change to JsonPath, the map keys in path containing periods (.) or special characters (/) are now supported with the use of "":

- equal:
    path: metadata.annotations["kubernetes.io/ingress.class"]
    value: nginx

In the next releases, it will be possible to validate multiple paths when JsonPath results in multiple results.

DocumentSelector

The test job or assertion can also specify a documentSelector rather than a documentIndex. Note that the documentSelector will always override a documentIndex if a match is found. This field is particularly useful when helm produces multiple templates and the order is not always guaranteed.

The path in the documentSelector has Yaml JsonPath Support, using JsonPath expressions it is possible to filter on multiple fields.

The value in the documentSelector can validate complete yaml objects and is optional.

...
tests:
  - it: should pass
    values:
      - ./values/staging.yaml
    set:
      image.pullPolicy: Always
      resources:
        limits:
          memory: 128Mi
    template: deployment.yaml
    documentSelector:
      path: metadata.name
      value: my-service-name
    asserts:
      - equal:
          path: metadata.name
          value: my-deploy

Example

Check test/data/v3/basic/ for some basic use cases of a simple chart.

Open Source Community Examples

Open-source solutions that uses helm-unittest to improve helm and kubernetes experience

Snapshot Testing

Sometimes you may just want to keep the rendered manifest not changed between changes without every details asserted. That's the reason for snapshot testing! Check the tests below:

templates:
  - templates/deployment.yaml
tests:
  - it: pod spec should match snapshot
    asserts:
      - matchSnapshot:
          path: spec.template.spec
  # or you can snapshot the whole manifest
  - it: manifest should match snapshot
    asserts:
      - matchSnapshot: {}
  - it: manifest should match snapshot and pattern and not match another pattern
    asserts:
      - matchSnapshot:
          matchRegex:
            pattern: .*app.*
          notMatchRegex:
              pattern: .*bcde.*

The matchSnapshot assertion validates the content rendered the same as cached last time. It fails if the content has changed, and you should check and update the cache with -u, --update-snapshot option of cli.

helm unittest -u my-chart

The cache files are stored as __snapshot__/*_test.yaml.snap at the directory your test file placed, you should add them in version control with your chart.

Dependent subchart Testing

If you have hard dependency subcharts (installed via helm dependency) existing in charts directory (they don't need to be extracted), it is possible to unittest these from the root chart. This feature can be helpful to validate if good default values are accidentally overwritten within your default helm chart.

# $YOUR_CHART/tests/xxx_test.yaml
templates:
  - charts/postgresql/templates/xxx.yaml
tests:
  - it:
    set:
      # this time required to prefix with "postgresql."
      postgresql.somevalue: should_be_scoped
    asserts:
      - ...

Note 1: if dependent subcharts uses an alias, use the alias name in the templates. Note 2: using the folder structure in templates can also be used to unittest templates which are placed in subfolders or unittest subcharts from the rootchart.

Check test/data/v3/with-subchart/ as an example.

Tests within subchart

If you have customized hard dependency subcharts (not installed via helm dependency, but added manually) existing in charts directory, tests inside would also be executed by default. You can disable this behavior by setting --with-subchart=false flag in cli, thus only the tests in root chart will be executed. Notice that the values defined in subchart tests will be automatically scoped, you don't have to add dependency scope yourself:

# with-subchart/charts/child-chart/tests/xxx_test.yaml
templates:
  - templates/xxx.yaml
tests:
  - it:
    set:
      # no need to prefix with "child-chart."
      somevalue: should_be_scoped
    asserts:
      - ...

Check test/data/v3/with-subchart/ as an example.

Test Suite code completion and validation

Most popular IDEs (IntelliJ, Visual Studio Code, etc.) support applying schemas to YAML files using a JSON Schema. This provides comprehensive documentation as well as code completion while editing the test-suite file:

Code completion

In addition, test-suite files can be validated while editing so incorrectly added additional properties or incorrect data types can be detected while editing:

Code Validation

Visual Studio Code

When developing with VSCode, the very popular YAML plug-in (created by RedHat) allows adding references to schemas by adding a comment line on top of the file:

# yaml-language-server: $schema=https://raw.githubusercontent.com/helm-unittest/helm-unittest/main/schema/helm-testsuite.json
suite: http-service.configmap_test.yaml
templates: [configmap.yaml]
release:
  name: test-release
  namespace: TEST_NAMESPACE

Alternatively, you can add the schema globally to the IDE, using a well defined pattern:

"yaml.schemas": {
  "https://raw.githubusercontent.com/helm-unittest/helm-unittest/main/schema/helm-testsuite.json": ["charts/*/tests/*_test.yaml"]
}

IntelliJ

Similar to VSCode, IntelliJ allows mapping file patterns to schemas via preferences: Languages & Frameworks -> Schemas and DTDs -> JSON Schema Mappings

Add Json Schema

Frequently Asked Questions

As more people use the unittest plugin, more questions will come. Therefore a Frequently Asked Question page is created to answer the most common questions.

If you are missing an answer to a question, feel free to raise a ticket.

Related Projects / Commands

This plugin is inspired by helm-template, and the idea of snapshot testing and some printing formats come from jest.

And there are some other helm commands you might want to use:

  • helm template: render the chart and print the output.

  • helm lint: examines a chart for possible issues, useful to validate chart dependencies.

  • helm test: test a release with testing pod defined in chart. Note this does create resources on your cluster to verify if your release is correct. Check the doc.

Alternatively, you can also use generic tests frameworks:

License

MIT

Contributing

Issues and PRs are welcome! To start developing this plugin please follow the Contribution guidelines.

About

BDD styled unit test framework for Kubernetes Helm charts as a Helm plugin.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1.4k stars

Watchers

9 watching

Forks

Releases

Packages

Used by

Contributors

Languages