Airflow Web Interface Crashes with 500 Error (FAB Auth, MySQL Timeout) #57859

Apache Airflow version

3.1.1

If "Other Airflow 2/3 version" selected, which one?

No response

What happened?

Issue Summary

I'm facing an issue with the Airflow web interface authentication since upgrading Airflow from v3.0.6 to v3.1.0. The problem still persists on v3.1.1.

I’m using the FAB (Flask AppBuilder) auth manager for authentication, and I suspect the issue is related to it.

Problem Description

After the upgrade, the Airflow web interface intermittently becomes inaccessible. The issue typically appears:

• After a day of normal operation, or
• When a second user logs in to Airflow from another device.

When the issue occurs, attempting to access the Airflow web UI results in a 500 Internal Server Error.
Restarting the Airflow webserver temporarily resolves the issue, but it reappears after a few hours.

Troubleshooting Performed

• Tried switching database libraries from pymysql to mysqldb.
• Verified the database connection and Airflow configuration.
• Observed that the issue correlates with database connection or session handling errors (example log below).

Recent Log Snippet

sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (4031, 'The client was disconnected by the server because of inactivity. See wait_timeout and interactive_timeout for configuring this behavior.') [SQL: SELECT ab_user.id AS ab_user_id, ab_user.first_name AS ab_user_first_name, ab_user.last_name AS ab_user_last_name, ab_user.username AS ab_user_username, ab_user.password AS ab_user_password, ab_user.active AS ab_user_active, ab_user.email AS ab_user_email, ab_user.last_login AS ab_user_last_login, ab_user.login_count AS ab_user_login_count, ab_user.fail_login_count AS ab_user_fail_login_count, ab_user.created_on AS ab_user_created_on, ab_user.changed_on AS ab_user_changed_on, ab_user.created_by_fk AS ab_user_created_by_fk, ab_user.changed_by_fk AS ab_user_changed_by_fk FROM ab_user WHERE ab_user.id = %s] [parameters: (1,)]

After the above error, I get the following error for any attempts in login to airflow web interface.

sqlalchemy.exc.PendingRollbackError: Can't reconnect until invalid transaction is rolled back. (Background on this error at: https://sqlalche.me/e/14/8s2b)

What you think should happen instead?

Airflow web interface should remain accessible and handle multiple user logins without requiring frequent webserver restarts.

How to reproduce

It usually comes after a day or if a second user logins from the another device. After that, I am unable to access airflow web interface.

Operating System

Red Hat Enterprise Linux 9.6

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

Web interface returns 500 Internal Server Error after some time or when multiple users log in.
Only restarting the webserver restores functionality temporarily.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct