add values to specify resources on init-containers #684
Conversation
Signed-off-by: fogelman <[email protected]>
thesuperzapper
left a comment
thesuperzapper
left a comment
There was a problem hiding this comment.
@fogelman This PR does not work as it currently stands (see comments from the review).
But either way, I think we should use a different approach, and have global resource values for each init-container type. This is because it does not make sense to give the worker/scheduler resources to the install_pip_packages, wait_for_db_migrations, or check_db init-containers, as they very likely do not need 2 CPU / 4GB RAM, for example.
These values might look like:
airflow:
## << put these configs JUST BEFORE the `airflow.localSettings` in the default values.yaml >>
## configs for the airflow init-containers
##
initContainers:
## configs for the `check-db` init-container
checkDb:
## resource requests/limits for the `check-db` init-container
## - spec for ResourceRequirements:
## https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#resourcerequirements-v1-core
##
resources: {}
## configs for the `wait-for-db-migrations` init-container
waitForDbMigrations:
## resource requests/limits for the `wait-for-db-migrations` init-container
## - spec for ResourceRequirements:
## https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#resourcerequirements-v1-core
##
resources: {}
## configs for the `install-pip-packages` init-container
installPipPackages:
## resource requests/limits for the `install-pip-packages` init-container
## - spec for ResourceRequirements:
## https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#resourcerequirements-v1-core
##
resources: {}This also lets us add values to disable these init-containers in future, if people like.
| - name: check-db | ||
| {{- include "airflow.image" . | indent 2 }} | ||
| resources: | ||
| {{- toYaml .resources | nindent 4 }} |
There was a problem hiding this comment.
The value .resources will not be defined, you must also change every time the "airflow.init_container.check_db" template is called to include the appropriate resources.
For example, in the scheduler this might look like:
# top of file
{{- $resources := .Values.scheduler.resources }}
# in the init container section
{{- include "airflow.init_container.check_db" (dict "Release" .Release "Values" .Values "volumeMounts" $volumeMounts "resources" $resources) | indent 8 }}
thesuperzapper
changed the title
2 participants
What issues does your PR fix?
What does your PR do?
Allows users to specify resources for initContainers.
Checklist
For all Pull Requests
For releasing ONLY