Skip to content

Warning about unsafe migrations #10129

@mik-laj

Description

@mik-laj

Helllo,

We currently have two migrations that cannot always be successful, which can lead to database corruption. All modern RDMSs should handle this case correctly and roll back the migrations, but ... it would be better if we detected these situations before starting the migration and not run the migration if we know that the operation will fail. This will also allow you to display much more user-friendly messages.
https://github.com/apache/airflow/blob/master/UPDATING.md#database-schema-changes

A similar check should be part of the airflow upgrade-check, but this command will be available in Airflow 1.10 and will not be run before migration starts.

Therefore, I propose that we write two small functions that will detect these two cases before starting the migration.
https://github.com/apache/airflow/blob/master/airflow/utils/db.py#L612
When this happens, we should display a message to the end-user along with an instruction describing the manual steps they have to do on their side.
Please note that if we do not have a table in the database, we should allow the migration to run. In other words, we should handle the case where the table does not exist in the database, so we cannot check for problematic database entries.

Best regards,
Kamil Breguła

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions