Repository navigation
Closed
Labels
area:APIAirflow's REST/HTTP APIAirflow's REST/HTTP APIarea:autharea:corekind:bugThis is a clearly a bugThis is a clearly a bugpriority:highHigh priority bug that should be patched quickly but does not require immediate new releaseHigh priority bug that should be patched quickly but does not require immediate new release
Description
Apache Airflow version
3.1.1
If "Other Airflow 2/3 version" selected, which one?
No response
What happened?
History begins here.
Log in / log out works right after deploy, but after some time, it starts failing (log in is impossible).
Re-creation of the API server k8s pod helps - it starts working again.
API Server log (several attempts to log in):
INFO: 172.25.119.245:58858 - "GET /api/v2/version HTTP/1.1" 200 OK
2025-10-30T07:07:47.868081Z [error [] Exception on /oauth-authorized/azure [GET[] [airflow.providers.fab.www.app] loc=app.py:1744
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
cursor.execute(statement, parameters)
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 179, in execute
res = self._query(mogrified_query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 330, in _query
db.query(q)
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/connections.py", line 280, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (2006, 'Server has gone away')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/security/views.py", line 713, in oauth_authorized
user = self.appbuilder.sm.auth_user_oauth(userinfo)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 2031, in auth_user_oauth
user = self.find_user(username=username)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1426, in find_user
return self.session.scalars(
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2459, in scalars
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
return meth(
^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
self._handle_dbapi_exception(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
cursor.execute(statement, parameters)
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 179, in execute
res = self._query(mogrified_query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 330, in _query
db.query(q)
File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/connections.py", line 280, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'Server has gone away')
[SQL: SELECT ab_user.id, ab_user.first_name, ab_user.last_name, ab_user.username, ab_user.password, ab_user.active, ab_user.email, ab_user.last_login, ab_user.login_count, ab_user.fail_login_count, ab_user.created_on, ab_user.changed_on, ab_user.created_by_fk, ab_user.changed_by_fk
FROM ab_user
WHERE lower(ab_user.username) = lower(%s)]
[parameters: ('4dac55b4-dc30-42f5-8e4f-4846ab968868',)]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
INFO: 172.25.120.253:55062 - "GET /auth/oauth-authorized/azure?code=1.AQMA9KHHarFf....dacd5d2b HTTP/1.1" 500 Internal Server Error
INFO: 172.25.120.253:55062 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO: 172.25.119.245:38882 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:38884 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:33248 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:33260 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:35764 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:35760 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:39466 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:39468 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:33598 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:33584 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:59828 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:59826 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:32796 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:32790 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:53710 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:53712 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36588 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36590 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:34038 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:34036 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36746 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36744 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:37786 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:37788 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:50022 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:50036 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:48980 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:48976 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.120.253:41384 - "GET / HTTP/1.1" 200 OK
INFO: 172.25.120.253:41384 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41384 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.117.209:50626 - "GET /ui/auth/menus HTTP/1.1" 401 Unauthorized
INFO: 172.25.117.209:50630 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41388 - "GET /api/v2/plugins HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41384 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.117.209:50626 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.120.253:41384 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.120.253:41384 - "GET /auth/login/?next=https://<our_airflow_host> HTTP/1.1" 200 OK
INFO: 172.25.120.253:41408 - "GET /auth/static/appbuilder/css/fontawesome/solid.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/appbuilder/css/bootstrap.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/fontawesome/regular.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/fontawesome.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/brands.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/bootstrap-datepicker/bootstrap-datepicker3.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/v4-shims.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/select2/select2.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/appbuilder/css/select2/select2-bootstrap.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/flags/flags16.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/ab.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/airflowDefaultTheme.ff5a35f322070b094aa2.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/materialIcons.3e67dd6fbfcc4f3b5105.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/loadingDots.2e5f555f0753107b0300.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/dist/jquery-ui.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/dist/main.3cf3be1a0c5439bb640d.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/js/ab_filters.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/flash.5583a9e0cf11f2be93da.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/jquery-latest.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/ab_actions.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/appbuilder/js/bootstrap-datepicker/bootstrap-datepicker.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/js/bootstrap.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/select2/select2.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/ab.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/jquery-ui.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/dist/main.3cf3be1a0c5439bb640d.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/moment.9baee5ec3d7639a10897.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/webfonts/fa-brands-400.woff2 HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/login/azure?next=https%3A//<our_airflow_host>/ HTTP/1.1" 302 Found
2025-10-30T07:10:14.906063Z [error [] Exception on /oauth-authorized/azure [GET[] [airflow.providers.fab.www.app] loc=app.py:1744
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/security/views.py", line 713, in oauth_authorized
user = self.appbuilder.sm.auth_user_oauth(userinfo)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 2031, in auth_user_oauth
user = self.find_user(username=username)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1426, in find_user
return self.session.scalars(
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2459, in scalars
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
return meth(
^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1813, in _execute_context
conn = self._revalidate_connection()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 673, in _revalidate_connection
self._invalid_transaction()
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 663, in _invalid_transaction
raise exc.PendingRollbackError(
sqlalchemy.exc.PendingRollbackError: Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b)
INFO: 172.25.117.209:50632 - "GET /auth/oauth-authorized/azure?code=1.AQMA9KHH...mCp_QE&session_state=009a7329-bd24-d99e-9121-bb58dacd5d2b HTTP/1.1" 500 Internal Server Error
INFO: 172.25.117.209:50632 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO: 172.25.119.245:39872 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:39870 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:59258 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:59260 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:41694 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:41680 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:48448 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:48450 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.117.209:50632 - "GET / HTTP/1.1" 200 OK
INFO: 172.25.117.209:50632 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 172.25.117.209:50632 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.117.209:50632 - "GET /ui/config HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41402 - "GET /ui/auth/menus HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41388 - "GET /api/v2/plugins HTTP/1.1" 401 Unauthorized
INFO: 172.25.120.253:41384 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.117.209:50632 - "GET /auth/login/?next=https://<our_airflow_host>/ HTTP/1.1" 200 OK
INFO: 172.25.120.253:41384 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.120.253:41384 - "GET /api/v2/auth/login?next=https%3A%2F%2F<our_airflow_host>%2F HTTP/1.1" 307 Temporary Redirect
INFO: 172.25.120.253:41384 - "GET /auth/login/?next=https://<our_airflow_host>/ HTTP/1.1" 200 OK
INFO: 172.25.120.253:41384 - "GET /auth/static/appbuilder/css/bootstrap.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/fontawesome/regular.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/fontawesome.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/fontawesome/solid.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/bootstrap-datepicker/bootstrap-datepicker3.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/fontawesome/v4-shims.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/select2/select2.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/select2/select2-bootstrap.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/css/ab.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/css/flags/flags16.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/fontawesome/brands.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/dist/materialIcons.3e67dd6fbfcc4f3b5105.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/dist/main.3cf3be1a0c5439bb640d.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/dist/airflowDefaultTheme.ff5a35f322070b094aa2.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/dist/loadingDots.2e5f555f0753107b0300.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/dist/jquery-ui.min.css HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/appbuilder/js/bootstrap.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/dist/flash.5583a9e0cf11f2be93da.css HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/appbuilder/js/ab_actions.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/ab_filters.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/js/jquery-latest.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/appbuilder/js/ab.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50632 - "GET /auth/static/appbuilder/js/bootstrap-datepicker/bootstrap-datepicker.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.117.209:50626 - "GET /auth/static/appbuilder/js/select2/select2.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/dist/jquery-ui.min.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41402 - "GET /auth/static/dist/moment.9baee5ec3d7639a10897.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41384 - "GET /auth/static/dist/main.3cf3be1a0c5439bb640d.js HTTP/1.1" 304 Not Modified
INFO: 172.25.120.253:41388 - "GET /auth/static/appbuilder/css/webfonts/fa-brands-400.woff2 HTTP/1.1" 304 Not Modified
INFO: 172.25.119.245:56250 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:56248 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.120.253:41388 - "GET /auth/login/azure?next=https%3A//<our_airflow_host>/ HTTP/1.1" 302 Found
2025-10-30T07:11:01.580516Z [error [] Exception on /oauth-authorized/azure [GET[] [airflow.providers.fab.www.app] loc=app.py:1744
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/security/views.py", line 713, in oauth_authorized
user = self.appbuilder.sm.auth_user_oauth(userinfo)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 2031, in auth_user_oauth
user = self.find_user(username=username)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1426, in find_user
return self.session.scalars(
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2459, in scalars
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
return meth(
^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1813, in _execute_context
conn = self._revalidate_connection()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 673, in _revalidate_connection
self._invalid_transaction()
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 663, in _invalid_transaction
raise exc.PendingRollbackError(
sqlalchemy.exc.PendingRollbackError: Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b)
INFO: 172.25.120.253:41388 - "GET /auth/oauth-authorized/azure?code=1.AQMA9KHHarF...E2DyWwE-IXOp4&session_state=009a7329-bd24-d99e-9121-bb58dacd5d2b HTTP/1.1" 500 Internal Server Error
INFO: 172.25.120.253:41388 - "GET /favicon.ico HTTP/1.1" 200 OK
INFO: 172.25.119.245:40900 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:40898 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36386 - "GET /api/v2/version HTTP/1.1" 200 OK
INFO: 172.25.119.245:36388 - "GET /api/v2/version HTTP/1.1" 200 OK
What you think should happen instead?
No response
How to reproduce
Not sure when, but after several hours after the deployment of v3.1.1, it can be reproduced (but if the API server pod was restarted for some reason, the problem will not appear, need to wait).
Operating System
linux
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
Chart v1.18, deployed to AWS EKS, using Azuer OAuth.
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
area:APIAirflow's REST/HTTP APIAirflow's REST/HTTP APIarea:autharea:corekind:bugThis is a clearly a bugThis is a clearly a bugpriority:highHigh priority bug that should be patched quickly but does not require immediate new releaseHigh priority bug that should be patched quickly but does not require immediate new release
Activity
vincbeck commentedon Oct 30, 2025
Duplicate of #57065
coenvd commentedon Nov 3, 2025
We have the same issue on version 3.1.1. I thought I could work around it by refreshing with empty cache and hard reload in chrome, but this only worked once. I dont know what logs exactly to deliver in order to help you, but this is the full log that I get from the webserver when I try to log in. Please let me know what other logs I can deliver in order to help.
vincbeck commentedon Nov 3, 2025
I'll definitely prioritize this one since it is impacting multiple users. Hopefully I can reproduce it
ktaborski commentedon Nov 4, 2025
probably duplicate with:
#57470
git commit -m "Fixed provider FAB - duplicated SQLAlchemy sessions ca…
Fixed provider FAB - duplicated SQLAlchemy sessions caused transactio…
5 remaining items
zachliu commentedon Nov 12, 2025
looks like it's fixed by #58196
no longer need to reboot server every 24 hours after jwt expires 🙏🙏🙏
sadly not gonna be in 3.1.3 😿
potiuk commentedon Nov 12, 2025
Cool Closing. I plan to start next provider's release tomorrow.
potiuk commentedon Nov 12, 2025
You should be able to upgrade to newer FAB provider when it is relased.
Fix OAuth session race condition causing false 401 errors during login
Fix OAuth session race condition causing false 401 errors during login
Fix OAuth session race condition causing false 401 errors during login (
Fix OAuth session race condition causing false 401 errors during login (
Fix OAuth session race condition causing false 401 errors during login (
Fix OAuth session race condition causing false 401 errors during login (
Fix OAuth session race condition causing false 401 errors during login (
Fix OAuth session race condition causing false 401 errors during login (
Fix OAuth session race condition causing false 401 errors during login (
Fix OAuth session race condition causing false 401 errors during login (
Fix OAuth session race condition causing false 401 errors during login (