Skip to content

Fix wrapping of run() method result of exasol and snoflake DB hooks #27997

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
potiuk merged 1 commit into from
Nov 30, 2022
Merged

Fix wrapping of run() method result of exasol and snoflake DB hooks #27997

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions airflow/providers/exasol/hooks/exasol.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,19 +159,21 @@ def run(
"""
if isinstance(sql, str):
if split_statements:
sql = self.split_sql_string(sql)
sql_list: Iterable[str] = self.split_sql_string(sql)
Copy link
Member Author

@potiuk potiuk Nov 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The root cause of the problem is line https://github.com/apache/airflow/pull/27997/files#diff-00b215634013629b320a263630bf6948ed33c2a8528754fc78d69484e1320845R191

thesql parameter changed it type - when it was passed as "string", it was always converted to list and that caused the return to be wrapped in list as well. So when sql was used in return_single_query_results, it was already a list and return_single_query_resuts is always false when you pass list of queries.

else:
sql = [self.strip_sql_string(sql)]
sql_list = [self.strip_sql_string(sql)]
else:
sql_list = sql

if sql:
self.log.debug("Executing following statements against Exasol DB: %s", list(sql))
if sql_list:
self.log.debug("Executing following statements against Exasol DB: %s", list(sql_list))
else:
raise ValueError("List of SQL statements is empty")

with closing(self.get_conn()) as conn:
self.set_autocommit(conn, autocommit)
results = []
for sql_statement in sql:
for sql_statement in sql_list:
with closing(conn.execute(sql_statement, parameters)) as cur:
self.log.info("Running statement: %s, parameters: %s", sql_statement, parameters)
if handler is not None:
Expand Down
14 changes: 9 additions & 5 deletions airflow/providers/snowflake/hooks/snowflake.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -353,12 +353,16 @@ def run(
if isinstance(sql, str):
if split_statements:
split_statements_tuple = util_text.split_statements(StringIO(sql))
sql = [sql_string for sql_string, _ in split_statements_tuple if sql_string]
sql_list: Iterable[str] = [
sql_string for sql_string, _ in split_statements_tuple if sql_string
]
else:
sql = [self.strip_sql_string(sql)]
sql_list = [self.strip_sql_string(sql)]
else:
sql_list = sql

if sql:
self.log.debug("Executing following statements against Snowflake DB: %s", list(sql))
if sql_list:
self.log.debug("Executing following statements against Snowflake DB: %s", sql_list)
else:
raise ValueError("List of SQL statements is empty")

Expand All @@ -368,7 +372,7 @@ def run(
# SnowflakeCursor does not extend ContextManager, so we have to ignore mypy error here
with closing(conn.cursor(DictCursor)) as cur: # type: ignore[type-var]
results = []
for sql_statement in sql:
for sql_statement in sql_list:
self._run_command(cur, sql_statement, parameters)

if handler is not None:
Expand Down