Skip to content

Commit 4202418

Browse files
dstandishuranusjr
andauthored
Remove dag_code records with no serdag (#49478)
It is assumed by the migration process that there will not be any dag_code records that don't have a serdag record. It seems this only happens when there are two files with the same dag id. --------- Co-authored-by: Tzu-ping Chung <[email protected]>
1 parent 49181cf commit 4202418

3 files changed

Lines changed: 9 additions & 1 deletion

File tree

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
bc93e7288a7a8355b15dc721accaf80260f370f3afa0d478248f9fe4692a1f1d
1+
2a3ce828be7c3800cb61de4c081fcf46f1f94e31ac732f831e65a8a9581b5bb0

‎airflow-core/docs/installation/upgrading_to_airflow3.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,11 @@ Step 2: Clean and back up your existing Airflow Instance
4141
upgrade process. These schema changes can take a long time if the database is large. For a faster, safer migration, we recommend that you clean up your Airflow meta-database before the upgrade.
4242
You can use the ``airflow db clean`` :ref:`Airflow CLI command<cli-db-clean>` to trim your Airflow database.
4343

44+
- Ensure that there are no errors related to dag processing, such as ``AirflowDagDuplicatedIdException``. You should
45+
be able to run ``airflow dags reserialize`` with no errors. If you have have to resolve errors from dag processing,
46+
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
47+
(and all errors gone) before you proceed with upgrade.
48+
4449

4550
Step 3: DAG Authors - Check your Airflow DAGs for compatibility
4651
----------------------------------------------------------------

‎airflow-core/src/airflow/migrations/versions/0047_3_0_0_add_dag_versioning.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,9 @@ def _airflow_2_fileloc_hash(fileloc):
6666
def upgrade():
6767
"""Apply add dag versioning."""
6868
conn = op.get_bind()
69+
70+
op.execute("DELETE FROM dag_code WHERE fileloc_hash NOT IN (SELECT fileloc_hash FROM serialized_dag)")
71+
6972
op.create_table(
7073
"dag_version",
7174
sa.Column("id", UUIDType(binary=False), nullable=False),

0 commit comments

Comments
 (0)