Skip to content

Validate Helm values.yaml #10664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
kaxil merged 1 commit into from
Sep 21, 2020
Merged

Validate Helm values.yaml #10664

kaxil merged 1 commit into from
Sep 21, 2020

Conversation

@flvndh
Copy link
Contributor

@flvndh flvndh commented Aug 31, 2020

Tackle #10634

@boring-cyborg boring-cyborg bot added the area:helm-chart Airflow Helm Chart label Aug 31, 2020
@mik-laj
Copy link
Member

mik-laj commented Aug 31, 2020

We have some errors Can you look at it?

2020-08-31T16:29:32.9918043Z helm install airflow . --namespace airflow --set defaultAirflowRepository=apache/airflow --set images.airflow.repository=apache/airflow --set images.airflow.tag=master-python3.6-kubernetes -v 1 --set defaultAirflowTag=master-python3.6-kubernetes -v 1 --set config.api.auth_backend=airflow.api.auth.backend.default
2020-08-31T16:29:33.1466658Z Error: values don't meet the specifications of the schema(s) in the following chart(s):
2020-08-31T16:29:33.1467262Z airflow:
2020-08-31T16:29:33.1467809Z - fernetKey: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1468655Z - statsd.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1469091Z - registry.secretName: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1469800Z - rbacEnabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1470663Z - elasticsearch.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1471205Z - elasticsearch.secretName: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1471649Z - cleanup.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1472085Z - allowPodLaunching: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1472753Z - postgresql.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1473407Z - data.metadataSecretName: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1473915Z - data.resultBackendSecretName: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1474567Z - data.metadataConnection.host: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1475291Z - data.resultBackendConnection.host: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1475828Z - pgbouncer.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1476508Z - pgbouncer.podDisruptionBudget.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1476875Z - fernetKeySecretName: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1477513Z - ingress.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1478153Z - ingress.web.tls.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1478797Z - ingress.flower.tls.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1479444Z - workers.persistence.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1480094Z - workers.persistence.storageClassName: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1480871Z - workers.persistence.fixPermissions: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1484488Z - workers.safeToEvict: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1492524Z - workers.keda.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1499109Z - scheduler.podDisruptionBudget.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1499838Z - scheduler.airflowLocalSettings: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1500356Z - scheduler.safeToEvict: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1501174Z - dags.persistence.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1501673Z - dags.persistence.storageClassName: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1502131Z - dags.persistence.existingClaim: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1502553Z - dags.gitSync.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1502934Z - networkPolicies.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1503347Z - images.flower.tag: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1503757Z - images.flower.repository: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1504173Z - webserver.webserverConfig: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1504606Z - webserver.defaultUser.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1505087Z - redis.password: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1505510Z - redis.safeToEvict: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1505936Z - redis.persistence.enabled: Invalid type. Expected: integer, given: boolean
2020-08-31T16:29:33.1506365Z - redis.persistence.storageClassName: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1506783Z - redis.passwordSecretName: Invalid type. Expected: string, given: null
2020-08-31T16:29:33.1507330Z - redis.brokerURLSecretName: Invalid type. Expected: string, given: null

mik-laj
mik-laj reviewed Aug 31, 2020
View reviewed changes
@ryw ryw requested a review from dimberman September 2, 2020 12:45
mik-laj
mik-laj reviewed Sep 3, 2020
View reviewed changes
mik-laj
mik-laj reviewed Sep 3, 2020
View reviewed changes
chart/values.schema.json Outdated
Copy link
Member

@mik-laj mik-laj Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here similar, but a little more complex. We expect there will be a map, which has a map that contains arbitrary text strings. Map<String, Map<String, String>

Copy link
Contributor Author

@flvndh flvndh Sep 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I do a

helm install airflow . --namespace airflow --set defaultAirflowRepository=apache/airflow --set images.airflow.repository=apache/airflow --set images.airflow.tag=master-python3.6-kubernetes -v 1 --set defaultAirflowTag=master-python3.6-kubernetes -v 1 --set config.api.auth_backend=airflow.api.auth.backend.default --dry-run

I get the following errors:

Error: values don't meet the specifications of the schema(s) in the following chart(s):
airflow:
- config.elasticsearch_configs.max_retries: Invalid type. Expected: string, given: integer
- config.elasticsearch_configs.timeout: Invalid type. Expected: string, given: integer
- config.scheduler.run_duration: Invalid type. Expected: string, given: integer
- config.scheduler.scheduler_heartbeat_sec: Invalid type. Expected: string, given: integer
- config.scheduler.statsd_port: Invalid type. Expected: string, given: integer

And indeed we have the following values:

config:
  scheduler:
    scheduler_heartbeat_sec: 5
    statsd_port: 9125
    run_duration: 41460
  elasticsearch_configs:
    max_retries: 3
    timeout: 30

Should we allow primitive types as well (integer, number and boolean) ?

Copy link
Member

@mik-laj mik-laj Sep 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We should allow other types also.

mik-laj
mik-laj reviewed Sep 3, 2020
View reviewed changes
mik-laj
mik-laj reviewed Sep 3, 2020
View reviewed changes
mik-laj
mik-laj reviewed Sep 3, 2020
View reviewed changes
@mik-laj
Copy link
Member

mik-laj commented Sep 3, 2020

It would be fantastic if we could generate documentation for README.md based on this, but we should do it in a separate PR. It has a lot of potential.

@flvndh
Copy link
Contributor Author

flvndh commented Sep 7, 2020

It would be fantastic if we could generate documentation for README.md based on this, but we should do it in a separate PR. It has a lot of potential.

Indeed, this is a great way to keep both in sync. Let me know if I can help with this as well :)

@flvndh flvndh marked this pull request as ready for review September 7, 2020 14:39
@flvndh
Copy link
Contributor Author

flvndh commented Sep 17, 2020

@mik-laj @dimberman Do you have any feedback on this PR ? Also, should I clean a bit the commit history ?

@mik-laj
Copy link
Member

mik-laj commented Sep 17, 2020

Also, should I clean a bit the commit history ?

We use squash and merge so it's not important to us. If you wish, you can do so, but it is not required.

I can see your branch is a bit outdated. Can you do a rebase?

@mik-laj
Copy link
Member

mik-laj commented Sep 17, 2020 •
edited
Loading

@dimberman Can you look at it? This is a nice improvement for the Helm Chart.

@flvndh flvndh force-pushed the issue/10634/validate-k8s-values-schema branch from 147744a to c4bb235 Compare September 17, 2020 18:21
@mik-laj
Copy link
Member

mik-laj commented Sep 18, 2020

@schnie Can I ask for review?

@mik-laj mik-laj mentioned this pull request Sep 21, 2020
Merged
@kaxil kaxil changed the title Validate values.yaml Validate Helm values.yaml Sep 21, 2020
@kaxil kaxil merged commit a449b1f into apache:master Sep 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kaxil kaxil kaxil approved these changes

@mik-laj mik-laj mik-laj approved these changes

@dimberman dimberman Awaiting requested review from dimberman

Assignees

No one assigned

Labels

area:helm-chart Airflow Helm Chart

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@flvndh @mik-laj @kaxil