Description

when transfer data from rdb to gcs, some column should be droped for security issue

Use case/motivation

When retrieving data from each table, mainly query to get all columns as shown below.

select * from table 

we can use specify the columns to import for each table, but in this case, we have to know the schema of all tables.
By adding exclude_column args to the operator as shown below, data from various tables can be sent to gcs with the same query.

to_tcs_task = MySQLToGCSOperator(
                    task_id='',
                    sql=f'select * from target_table,
                    bucket='',
                    filename='',
                    schema_filename='',
                    export_format='json',
                    exclude_column=exclude_column,
                    mysql_conn_id=source_connect_id,
                    gcp_conn_id=target_connect_id
                )

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct