dbapi_hook causes mysql error 2014 on large input #459
Description
This is the exception:
ProgrammingError: (2014, "Commands out of sync; you can't run this command now")
2015-09-24 14:54:41,780 - root - ERROR - (2014, "Commands out of sync; you can't run this command now")
Traceback (most recent call last):
File "/home/mtustin/airflowvenv/bin/airflow", line 10, in <module>
args.func(args)
File "/home/mtustin/airflowvenv/lib/python2.7/site-packages/airflow/bin/cli.py", line 204, in test
ti.run(force=True, ignore_dependencies=True, test_mode=True)
File "/home/mtustin/airflowvenv/lib/python2.7/site-packages/airflow/models.py", line 928, in run
result = task_copy.execute(context=context)
File "/home/mtustin/airflowvenv/lib/python2.7/site-packages/airflow/operators/mysql_operator.py", line 33, in execute
hook.run(self.sql)
File "/home/mtustin/airflowvenv/lib/python2.7/site-packages/airflow/hooks/dbapi_hook.py", line 100, in run
conn.commit()
_mysql_exceptions.ProgrammingError: (2014, "Commands out of sync; you can't run this command now")
This is the task which is triggering it:
refresh_db = MySqlOperator(
mysql_conn_id='clean_db_export',
sql='templates/dbschema.sql',
task_id='refresh_db',
dag=clean_and_export_dag)
templates/dbschema.sql is the result of running mysqldump --no-data.
I am able to successfully perform - in a separate task - a 4 line sql script which drops and recreates the database involved here.
The exception does not occur if I switch the order of these two lines: https://github.com/airbnb/airflow/blob/master/airflow/hooks/dbapi_hook.py#L100-L101 to give:
cur.close()
conn.commit()
I am running airflow and mysql on Redhat with kernel 3.10.0-229.el7.x86_64 #1 SMP.
I expect to have a PR open in the course of the day with that change.