Skip to content

Add SqlExecuteQueryOperator #25259

@eladkal

Description

@eladkal

Body

There has been previous attempt to add this operator in #10011
But now after #23971 is completed this should be much easier as DbApiHook has unified run method that works for all providers (with some configuration of parameters like split_statements etc..)

We can now explore again the option of SqlExecuteQueryOperator
There are two approaches that I can think of:

  1. Creating a BaseSqlExecuteQueryOperator that all Sql related operators (PostgresOperator, MySqlOperator, SnowflakeOperator) will inherit from.
  2. Deprecate the current individual operators in favor of SqlExecuteQueryOperator (just like Add SQLExecuteQueryOperator operator #10011 attempted to do)

Personally I prefer to try the 2nd approach first (It should be similar to what we do in SqlSensor or SqlBranchOperator the issue to resolve is how to handle the batch queries in some of the DBs and that is now should be possible thanks to #23971 in SqlSensor / SqlBranchOperator executing batch queries was never an issue as they are expected to execute only 1 SQL statement )

FYI @potiuk @kazanzhy

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions