Skip to content

Fix value precedence#10110

Merged
mattfarina merged 7 commits into
helm:mainfrom
zegerius:main
Apr 12, 2022
Merged

Fix value precedence#10110
mattfarina merged 7 commits into
helm:mainfrom
zegerius:main

Conversation

@zegerius

@zegerius zegerius commented Sep 7, 2021

Copy link
Copy Markdown
Contributor

Fix value precedence when importing values from child, as described in issue #9940

What this PR does / why we need it:

Closes #9940

If applicable:

  • this PR contains documentation
  • this PR contains unit tests
  • this PR has been tested for backwards compatibility

Fix value precedence when importing values from child. Closes helm#9940

Signed-off-by: Aram Zegerius <[email protected]>
@helm-bot helm-bot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Sep 7, 2021
@bacongobbler

Copy link
Copy Markdown
Member

Can you please provide a test case to confirm #9940 has been fixed?

@zegerius

zegerius commented Sep 7, 2021

Copy link
Copy Markdown
Contributor Author

@bacongobbler sure, I'll do that tomorrow together with my colleague @ankitabhopatkar13. This was tested against this repository https://github.com/ankitabhopatkar13/helm-values-issue. AFAIK no tests broke, I am not sure how often this code block is hit.

@bacongobbler bacongobbler added this to the 3.8.0 milestone Sep 7, 2021
Closes helm#9940

Signed-off-by: Aram Zegerius <[email protected]>
Signed-off-by: Ankita Bhopatkar <[email protected]>
@helm-bot helm-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 10, 2021
@zegerius

Copy link
Copy Markdown
Contributor Author

@bacongobbler - @ankitabhopatkar13 and I finalized the test. Output before and after the fix below:

Before fix

Running tool: /opt/homebrew/opt/go/libexec/bin/go test -timeout 30s -run ^TestProcessDependencyImportValuesMultiLevelPrecedence$ helm.sh/helm/v3/pkg/chartutil

--- FAIL: TestProcessDependencyImportValuesMultiLevelPrecedence (0.00s)
    /Users/aram/projects/helm/pkg/chartutil/dependencies_test.go:263: failed to match imported float value 9090 with expected 8080
FAIL
FAIL	helm.sh/helm/v3/pkg/chartutil	0.216s
FAIL

After fix

Running tool: /opt/homebrew/opt/go/libexec/bin/go test -timeout 30s -run ^TestProcessDependencyImportValuesMultiLevelPrecedence$ helm.sh/helm/v3/pkg/chartutil

ok  	helm.sh/helm/v3/pkg/chartutil	0.243s

@zonggen

zonggen commented Sep 10, 2021

Copy link
Copy Markdown
Member

Tested manually with chart from linked issue: https://github.com/ankitabhopatkar13/helm-values-issue

Before fix:

$ helm upgrade --install umbrella umbrella --debug --dry-run

MANIFEST:
---
# Source: umbrella/charts/app1/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: umbrella-app1
  labels:
    helm.sh/chart: app1-0.1.0
    app.kubernetes.io/name: app1
    app.kubernetes.io/instance: umbrella
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 1234
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: app1
    app.kubernetes.io/instance: umbrella
---
# Source: umbrella/charts/app2/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: umbrella-app2
  labels:
    helm.sh/chart: app2-0.1.0
    app.kubernetes.io/name: app2
    app.kubernetes.io/instance: umbrella
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 8080
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: app2
    app.kubernetes.io/instance: umbrella

$ helm upgrade --install umbrella umbrella --debug --dry-run --values umbrella/values.yaml

MANIFEST:
---
# Source: umbrella/charts/app1/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: umbrella-app1
  labels:
    helm.sh/chart: app1-0.1.0
    app.kubernetes.io/name: app1
    app.kubernetes.io/instance: umbrella
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 9090
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: app1
    app.kubernetes.io/instance: umbrella
---
# Source: umbrella/charts/app2/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: umbrella-app2
  labels:
    helm.sh/chart: app2-0.1.0
    app.kubernetes.io/name: app2
    app.kubernetes.io/instance: umbrella
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 9090
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: app2
    app.kubernetes.io/instance: umbrella

After fix:

$ helm upgrade --install umbrella umbrella --debug --dry-run

MANIFEST:
---
# Source: umbrella/charts/app1/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: umbrella-app1
  labels:
    helm.sh/chart: app1-0.1.0
    app.kubernetes.io/name: app1
    app.kubernetes.io/instance: umbrella
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 1234
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: app1
    app.kubernetes.io/instance: umbrella
---
# Source: umbrella/charts/app2/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: umbrella-app2
  labels:
    helm.sh/chart: app2-0.1.0
    app.kubernetes.io/name: app2
    app.kubernetes.io/instance: umbrella
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 8080
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: app2
    app.kubernetes.io/instance: umbrella

$ helm upgrade --install umbrella umbrella --debug --dry-run --values umbrella/values.yaml

MANIFEST:
---
# Source: umbrella/charts/app1/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: umbrella-app1
  labels:
    helm.sh/chart: app1-0.1.0
    app.kubernetes.io/name: app1
    app.kubernetes.io/instance: umbrella
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 1234
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: app1
    app.kubernetes.io/instance: umbrella
---
# Source: umbrella/charts/app2/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: umbrella-app2
  labels:
    helm.sh/chart: app2-0.1.0
    app.kubernetes.io/name: app2
    app.kubernetes.io/instance: umbrella
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 8080
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: app2
    app.kubernetes.io/instance: umbrella

Notice the helm upgrade --install umbrella umbrella --debug --dry-run --values umbrella/values.yaml does not set to library default port 9090 after the fix.

I think this fixes the issue and test case passed during local run.

@zegerius

Copy link
Copy Markdown
Contributor Author

I'll simplify the test data directory. There's a bit too many adjacent things going on there.

Matthew Fisher and others added 3 commits September 14, 2021 10:07
Problem: the warnings don't give enough details about which
values are problematic, only the name of the leaf key. This is
all the more annoying when you have a chart depending on other charts.

```
mainchart
 |
 +- subchart1
 +- subchart2
 +- subchart3
```

Here are some warnings I get before the change:
```
coalesce.go:199: warning: destination for credentials is a table. Ignoring non-table value
coalesce.go:160: warning: skipped value for resources: Not a table.
coalesce.go:160: warning: skipped value for googleSheetsServiceAccount: Not a table.
coalesce.go:199: warning: destination for googleSheetsServiceAccount is a table. Ignoring non-table value
coalesce.go:199: warning: destination for resources is a table. Ignoring non-table value []
coalesce.go:199: warning: destination for credentials is a table. Ignoring non-table value
coalesce.go:199: warning: destination for credentials is a table. Ignoring non-table value
coalesce.go:160: warning: skipped value for resources: Not a table.
coalesce.go:160: warning: skipped value for googleSheetsServiceAccount: Not a table.
```

with fix:
```
coalesce.go:162: warning: skipped value for subchart1.resources: Not a table.
coalesce.go:162: warning: skipped value for subchart2.googleSheetsServiceAccount: Not a table.
coalesce.go:211: warning: destination for subchart3.aws.credentials is a table. Ignoring non-table value ()
coalesce.go:211: warning: destination for mainchart.subchart3.aws.credentials is a table. Ignoring non-table value ()
coalesce.go:211: warning: destination for mainchart.subchart2.googleSheetsServiceAccount is a table. Ignoring non-table value ()
coalesce.go:211: warning: destination for mainchart.subchart1.resources is a table. Ignoring non-table value ([])
coalesce.go:162: warning: skipped value for subchart1.resources: Not a table.
coalesce.go:162: warning: skipped value for subchart2.googleSheetsServiceAccount: Not a table.
coalesce.go:211: warning: destination for subchart3.aws.credentials is a table. Ignoring non-table value ()
```

Signed-off-by: Damien Nozay <[email protected]>

add tests

Signed-off-by: Damien Nozay <[email protected]>
Signed-off-by: Aram Zegerius <[email protected]>
Signed-off-by: Ankita Bhopatkar <[email protected]>
@helm-bot helm-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 14, 2021
@zegerius

Copy link
Copy Markdown
Contributor Author

Simplified test data. I think it can be merged @bacongobbler @zonggen

@zonggen

zonggen commented Sep 14, 2021

Copy link
Copy Markdown
Member

Ran the test case and with helm upgrade. Looks good from my side.

@mattfarina mattfarina added the bug Categorizes issue or PR as related to a bug. label Jan 12, 2022
@mattfarina mattfarina modified the milestones: 3.8.0, 3.8.1 Jan 12, 2022
@ankitabhopatkar13

Copy link
Copy Markdown

Hi @mattfarina,

I see this has been marked to 3.8.1 milestone instead of the 3.8.0. Is there anything pending on this MR?

We have to workaround a lot because of this bug and would appreciate it released as soon as possible.

Thanks!

@mattfarina mattfarina modified the milestones: 3.8.1, 3.8.2 Mar 9, 2022

@hickeyma hickeyma left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the fix @zegerius.

@hickeyma
hickeyma requested a review from mattfarina April 7, 2022 09:01
@mattfarina
mattfarina merged commit 3fdb7ca into helm:main Apr 12, 2022
@mattfarina mattfarina added needs-pick Indicates that a PR needs to be cherry-picked into the next release candidate. picked Indicates that a PR has been cherry-picked into the next release candidate. labels Apr 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Categorizes issue or PR as related to a bug. needs-pick Indicates that a PR needs to be cherry-picked into the next release candidate. picked Indicates that a PR has been cherry-picked into the next release candidate. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Value files precedence behaviour changes when custom values file is used

8 participants