Description
In my company usage of Airflow, developmental instances of Airflow run on containerized PostgreSQL that are spawned at the same time the Airflow container is spawned. Before the Airflow container runs its initialization scripts, it needs to make sure that the PostgreSQL instance can be reached, for which airflow db check is a great option.
However, there is a non-deterministic race condition between the PSQL container and Airflow containers (not sure which will each readiness first and by how much), calling the airflow db check command once is not sufficient, and implementing a retry-timeout in shell script is feasible but unpleasant.
It would be great if the airflow db check command can take two additional optional arguments: --retry and --retry-delay (just like with curl) so that the database connection can be checked repeatedly for up to a specified number of times. This command should exit with 0 exit code if any of the retries succeeds, and 1 if all of the retries failed.
Use case/motivation
No response
Related issues
No response
Are you willing to submit a PR?
Code of Conduct
Description
In my company usage of Airflow, developmental instances of Airflow run on containerized PostgreSQL that are spawned at the same time the Airflow container is spawned. Before the Airflow container runs its initialization scripts, it needs to make sure that the PostgreSQL instance can be reached, for which
airflow db checkis a great option.However, there is a non-deterministic race condition between the PSQL container and Airflow containers (not sure which will each readiness first and by how much), calling the
airflow db checkcommand once is not sufficient, and implementing a retry-timeout in shell script is feasible but unpleasant.It would be great if the
airflow db checkcommand can take two additional optional arguments:--retryand--retry-delay(just like withcurl) so that the database connection can be checked repeatedly for up to a specified number of times. This command should exit with0exit code if any of the retries succeeds, and1if all of the retries failed.Use case/motivation
No response
Related issues
No response
Are you willing to submit a PR?
Code of Conduct