Apache Airflow Provider(s)

presto, trino

Versions of Apache Airflow Providers

apache-airflow-providers-trino==4.0.1
apache-airflow-providers-presto==4.0.1

Apache Airflow version

2.4.0

Operating System

macOS 12.6

Deployment

Docker-Compose

Deployment details

No response

What happened

When using the TrinoHook (PrestoHook also applies), only the get_records() and get_first() methods work as expected, the run() and insert_rows() do not.
The SQL statements sent by the problematic methods reach the database (visible in logs and UI), but they don't get executed.

The issue is caused by the hook not making the required subsequent requests to the Trino HTTP endpoints after the first request. More info here:

If the JSON document returned by the POST to /v1/statement does not contain a nextUri link, the query has completed, either successfully or unsuccessfully, and no additional requests need to be made. If the nextUri link is present in the document, there are more query results to be fetched. The client should loop executing a GET request to the nextUri returned in the QueryResults response object until nextUri is absent from the response.

SQL statements made by methods like get_records() do get executed because internally they call fetchone() or fetchmany() on the cursor, which do make the subsequent requests.

What you think should happen instead

The Hook is able to execute SQL statements other than SELECT.

How to reproduce

Connect to a Trino or Presto instance and execute any SQL statement (INSERT or CREATE TABLE) using TrinoHook.run(), the statements will reach the API but they won't get executed.

Then, provide a dummy handler function like this:
TrinoHook.run(..., handler=lambda cur: cur.description)

The description property internally iterates over the cursor requests, causing the statement getting executed.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct