feat(state): add mergeStrategy: fallback for first-file-wins env values by dschmidt · Pull Request #2578 · helmfile/helmfile

dschmidt · 2026-05-06T16:44:16Z

Summary

Adds an opt-in per-environment field mergeStrategy to EnvironmentSpec. When set to fallback, the precedence of files listed under values: is flipped: earlier files win, and later files only fill in missing keys. The default override (and an empty/unset field) keeps the historical last-file-wins behavior, so this PR is fully backwards compatible.

environments:
  production:
    mergeStrategy: fallback
    values:
      - cluster-specific.yaml      # wins on every key it defines
      - shared-defaults.yaml.gotmpl

Two coupled changes ship together because either alone is half-useful for the canonical use case:

Precedence flip in mapMerge. A naïve "drop mergo.WithOverride" approach is wrong because mergo's isEmptyValue would silently let enabled: true from a fallback file overwrite an explicit enabled: false from the winning file. Instead, the fallback path reuses maputil.MergeMaps (helmfile's existing deep-merge utility) with the new file as the merge base and the accumulator overlaid on top. The accumulator wins on every non-nil key, so explicit false, 0, "", empty list, and empty map values are preserved when set in an earlier file. Explicit null falls through to a later fallback (consistent with how MergeMaps treats nil from the override side everywhere else).
Cross-file template visibility under fallback. Under fallback strategy, .gotmpl values files now see values accumulated from earlier files in the same values: list via .Values. This makes derived defaults work natively:
```
# cluster-specific.yaml
cluster:
  domain: prod.example.com
```
```
# shared-defaults.yaml.gotmpl
service:
  domain: "service.{{ .Values.cluster.domain }}"   # → service.prod.example.com
```
Env CLI overrides and env values still win on overlap — the accumulator is layered under env.GetMergedValues() with mergo.WithOverride. Override mode keeps the historical template context unchanged.

Why

A common pattern is "one file with environment-specific values + one file with shared defaults". Today this is hard to express on two independent fronts: (a) the defaults file would override the specifics regardless of which order you list them in, and (b) .gotmpl values files cannot reference values from sibling files in the same values: list (the template context is just env.GetMergedValues() — the accumulator never feeds back in). Users currently work around both by hand-rolling a merged-values.yaml.gotmpl that uses readFile + tpl + manual merge to flip precedence and stage the template render. With mergeStrategy: fallback, the natural ordering matches the natural intent and the workaround disappears.

A worked example pattern that now works directly:

environments:
  prod:
    mergeStrategy: fallback
    values:
      - "environments/{{ .Environment.Name }}/*.yaml"   # env-specific overrides (may be empty)
      - environments/default/fallback.yaml              # shared defaults

Adding a new environment becomes mkdir environments/foo && touch environments/foo/values.yaml.

Out of scope

Changing secrets: merge behavior — stays override.
The top-level values: block — the one at the root of helmfile.yaml, separate from environments.*.values:. That list loads into Environment.Defaults and is then overridden by per-environment values regardless of strategy, so within-list merging there is a different (and much smaller) concern. A sibling root-level field could be added later if demand emerges, without breaking the per-env field shipped here.

dschmidt · 2026-05-06T17:03:28Z

A note on prior work — I should have checked harder before opening this. After the fact I went looking and found a pretty rich history I'd missed:

feat: add environments values inheritance #741 (the closest sibling — implements the cross-file template visibility piece, closed by stale-bot in December 2025)
discussion Release values inheritance #726 (the original RFC)
feat: add release values inheritance #725 (an earlier release-values attempt)
feat: accept strings that evaluates to bool in release condition #840 / @mumoshu's design comment in feat: add environments values inheritance #741

I arrived here from a different angle — the gotmpl readFile + tpl + merge workaround was annoying me, I went looking for where in the loader to fix it, and ended up writing this PR before realizing #741 had been chasing the same itch for two and a half years. Reading through it, what struck me was that there's no real disapproval of the feature itself — @mumoshu twice said he was looking forward to merging it. It stalled on review and eventually got auto-closed with a vague "outdated" note. I'd like to take another honest swing at it.

What this PR adds on top of where #741 ended up:

Fixes the explicit zero-value bug. feat: add environments values inheritance #741 leaves mergo.WithOverride in place, so enabled: true in a fallback file silently overwrites enabled: false in the winning file (mergo's isEmptyValue treats false/0/""/[] as empty). I switched the fallback path to a key-presence check so explicit zero values are preserved, with a dedicated test pinning each case.
Adds the precedence flip. feat: add environments values inheritance #741 makes later files visible to earlier ones via templates but doesn't flip merge order. The combination is what actually replaces the gotmpl workaround.
Smaller schema surface — one new field, no new list type.

On the alternative shape

I considered going with a separate sibling list to dodge the "values: gets a strategy switch" concern @mumoshu raised in #741:

environments:
  prod:
    values:
      - cluster-specific.yaml      # unchanged, last-wins
    fallbackValues:                # new field with these semantics
      - shared-defaults.yaml.gotmpl

Why I think this is harder for users

With one list and a strategy field, the rule is "read top-to-bottom; for fallback, earlier wins." That's it. With two parallel lists, a reader has to hold several questions in their head at once that the schema doesn't answer:

Which list wins when the same key appears in both? Is it always values: over fallbackValues:, or does it depend on order somehow?
Within fallbackValues:, what's the order? First-wins (matching the field name) or last-wins (matching the existing values: convention)?
Does fallbackValues: get processed as a continuation of values:, or as its own scope that's then layered under values: as a whole?
Do cross-file template references work the same in both lists, or only inside fallbackValues:?

Each of those has a reasonable answer, but you have to remember the answer for each. With the strategy-field shape, those questions don't arise — there's just one list with one rule.

The strategy field is also in the manifest, declarative, and explicit, so the same helmfile.yaml always renders the same way regardless of any external state. That's the property @mumoshu was protecting in #741 (no runtime flags changing manifest interpretation), and this shape preserves it.

@mumoshu @yxxhero @zhaque44 — would appreciate a review when you have time. Happy to iterate on naming or scope.

Copilot

Pull request overview

This PR adds an opt-in mergeStrategy field to environments.<name> that allows flipping environment values: precedence to “first-file-wins” (fallback) while also enabling later .gotmpl values files to reference accumulated values from earlier files via .Values.

Changes:

Introduces MergeStrategyOverride / MergeStrategyFallback and threads mergeStrategy through environment values loading.
Implements fallback merging using maputil.MergeMaps to preserve explicit non-nil zero values (e.g., false, 0, "", []) from earlier files.
Adds documentation and extensive unit tests + testdata covering fallback semantics and template visibility.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
pkg/state/state.go	Adds exported mergeStrategy constants.
pkg/state/environment.go	Adds `EnvironmentSpec.MergeStrategy` field and documents behavior.
pkg/state/envvals_loader.go	Adds mergeStrategy validation, fallback merge path, and fallback template-context enrichment.
pkg/state/create.go	Threads mergeStrategy from `EnvironmentSpec` into values loading.
pkg/app/desired_state_file_loader.go	Updates calls to `LoadEnvironmentValues` with new parameter.
pkg/state/envvals_loader_test.go	Adds comprehensive tests for fallback/override/default behavior and invalid values.
pkg/state/testdata/mergestrategy/*	New fixtures for chaining, zero/nil behavior, and gotmpl cross-file visibility.
docs/values-and-merging.md	Documents `mergeStrategy: fallback` behavior and examples.
docs/environments.md	Documents environment-level mergeStrategy usage and links to details.

dschmidt · 2026-05-07T06:18:23Z

We don't need to do it now, but should the need arise we can also use yaml tags like !override or !reset to prevent fallbacks from later specified files.

Copilot

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Introduces a per-environment mergeStrategy with valid values "override" (default, current behavior) and "fallback". This commit only adds the field, the constants, and a parse-time validator; the loader still ignores the value, so behavior is unchanged. Subsequent commits thread the value through the values loader and implement the fallback semantics. Signed-off-by: Dominik Schmidt <[email protected]>

Adds a mergeStrategy string parameter to LoadEnvironmentValues, loadValuesEntries, and mapMerge so the value can flow from EnvironmentSpec down to the merge call site. Behavior is unchanged in this commit; mapMerge ignores the strategy and the next commit implements the fallback semantics. Top-level state.DefaultValues and the --state-values-file/-set loaders are passed an empty strategy ("") since they have no per-environment spec to consult and stay on the default override behavior. Signed-off-by: Dominik Schmidt <[email protected]>

Adds a hand-rolled fallbackDeepMerge that, unlike mergo, preserves keys present in the destination even when their value is the zero value (false, 0, "", nil, empty list/map). mapMerge dispatches to it when mergeStrategy == "fallback"; "override" and the empty default keep using mergo with WithOverride so existing behaviour is unchanged. Validation lives at the entry of LoadEnvironmentValues so a single chokepoint guards the field. Invalid values produce an error naming both the offending value and the valid options. Tests cover: first-file-wins precedence, gap filling, deep nested merge, three-file chains, explicit zero-value preservation (the case naïve mergo gets wrong), explicit nil preservation, inline map entries, override regression, default-equals-override equivalence, and invalid-strategy errors. Signed-off-by: Dominik Schmidt <[email protected]>

Under mergeStrategy: fallback, .gotmpl values files can now reference values from earlier files in the same `values:` list via .Values (e.g. `service.domain: "service.{{ .Values.cluster.domain }}"`). The accumulated result is layered under env.GetMergedValues so env defaults, env values, and CLI overrides still win on overlap. Override mode keeps the historical template context — unchanged — so this is strictly opt-in via the mergeStrategy field. Together with the precedence flip from the previous commit, this lets users replace the brittle two-stage `merged-values.yaml.gotmpl` workaround with native helmfile syntax. Tests cover the headline cross-file template reference case and pin the override-mode contract that prior-file values stay invisible. Signed-off-by: Dominik Schmidt <[email protected]>

Adds a new section to values-and-merging.md describing the override vs fallback strategies, the explicit-zero-value preservation guarantee, and the cross-file template reference behavior. Adds a brief pointer to environments.md so users land on the new field from the environment values discussion. Signed-off-by: Dominik Schmidt <[email protected]>

Replaces the hand-rolled fallbackDeepMerge with a single call to maputil.MergeMaps, swapping its arguments so the accumulated dest wins over the new src file. Same first-file-wins semantic, fewer lines, and the fallback path now inherits the same slice merge strategies the rest of helmfile already uses. The one observable behavior shift is for explicit nil values: under fallback, nil in an earlier file no longer 'wins' over a non-nil value in a later file — instead it falls through (matching MergeMaps' rule that nil from the override side only fills missing keys). This is internally consistent: nil-overwrites is an mergo.WithOverride quirk that lives only in the override path. The renamed test NilFallsThroughToFallback pins the new behavior with a comment referencing the contrast with override mode (Issue1154). Signed-off-by: Dominik Schmidt <[email protected]>

dschmidt · 2026-05-07T14:06:29Z

oh wow, I didnt expect the PR to get merged so quickly - thanks a lot! <3

When can we expect a release with this feature?

dschmidt requested review from aditmeno, itscaro, mumoshu, xiaomudk, yxxhero and zhaque44 as code owners May 6, 2026 16:44

dschmidt force-pushed the feat/environment-merge-strategy-fallback branch 4 times, most recently from 1868ee6 to df4b7d9 Compare May 6, 2026 17:23

yxxhero requested a review from Copilot May 6, 2026 23:36

Copilot started reviewing on behalf of yxxhero May 6, 2026 23:36 View session

Copilot AI reviewed May 6, 2026

View reviewed changes

Comment thread pkg/state/create.go

Comment thread pkg/state/environment.go Outdated

Comment thread docs/values-and-merging.md Outdated

Comment thread docs/environments.md Outdated

dschmidt force-pushed the feat/environment-merge-strategy-fallback branch from df4b7d9 to d48426e Compare May 7, 2026 06:09

yxxhero requested a review from Copilot May 7, 2026 08:12

Copilot started reviewing on behalf of yxxhero May 7, 2026 08:13 View session

Copilot AI reviewed May 7, 2026

View reviewed changes

Comment thread pkg/state/envvals_loader.go Outdated

Comment thread pkg/state/create.go

dschmidt force-pushed the feat/environment-merge-strategy-fallback branch from d48426e to 39a0398 Compare May 7, 2026 08:59

yxxhero requested a review from Copilot May 7, 2026 09:07

Copilot started reviewing on behalf of yxxhero May 7, 2026 09:08 View session

Copilot AI reviewed May 7, 2026

View reviewed changes

Comment thread pkg/state/envvals_loader.go

dschmidt force-pushed the feat/environment-merge-strategy-fallback branch from 39a0398 to 7f7dbeb Compare May 7, 2026 09:35

dschmidt added 6 commits May 7, 2026 21:01

yxxhero force-pushed the feat/environment-merge-strategy-fallback branch from 7f7dbeb to d7803b6 Compare May 7, 2026 13:01

yxxhero approved these changes May 7, 2026

View reviewed changes

yxxhero merged commit 0139304 into helmfile:main May 7, 2026
16 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(state): add mergeStrategy: fallback for first-file-wins env values#2578

feat(state): add mergeStrategy: fallback for first-file-wins env values#2578
yxxhero merged 6 commits into
helmfile:mainfrom
dschmidt:feat/environment-merge-strategy-fallback

dschmidt commented May 6, 2026 •

edited

Loading

Uh oh!

dschmidt commented May 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dschmidt commented May 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

dschmidt commented May 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

dschmidt commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

Out of scope

Uh oh!

dschmidt commented May 6, 2026

On the alternative shape

Why I think this is harder for users

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dschmidt commented May 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

dschmidt commented May 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dschmidt commented May 6, 2026 •

edited

Loading