I always face the following error when I try to run a BigQuery query that accesses connected sheets, when I use impersonation_chain.
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py", line 2203, in run_query
job = self.insert_job(configuration=configuration, project_id=self.project_id)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/common/hooks/base_google.py", line 439, in inner_wrapper
return func(self, *args, **kwargs)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py", line 1571, in insert_job
job.result(timeout=timeout, retry=retry)
File "/opt/python3.8/lib/python3.8/site-packages/google/cloud/bigquery/job/query.py", line 1499, in result
do_get_result()
File "/opt/python3.8/lib/python3.8/site-packages/google/cloud/bigquery/job/query.py", line 1489, in do_get_result
super(QueryJob, self).result(retry=retry, timeout=timeout)
File "/opt/python3.8/lib/python3.8/site-packages/google/cloud/bigquery/job/base.py", line 728, in result
return super(_AsyncJob, self).result(timeout=timeout, **kwargs)
File "/opt/python3.8/lib/python3.8/site-packages/google/api_core/future/polling.py", line 137, in result
raise self._exception
google.api_core.exceptions.Forbidden: 403 Access Denied: BigQuery BigQuery: Permission denied while getting Drive credentials.
Apache Airflow version
main (development)
What happened
I always face the following error when I try to run a BigQuery query that accesses connected sheets, when I use
impersonation_chain.I think it's because it always uses a default scope:
https://www.googleapis.com/auth/cloud-platform. We can set scopes with Airflow connections (code). However, we cannot set scopes withimpersonation_chain.What you think should happen instead
I would like the operators and hooks to accept custom scope -
https://www.googleapis.com/auth/drivein this case.How to reproduce
impersonation_chain.Operating System
Linux
Versions of Apache Airflow Providers
No response
Deployment
Google Cloud Composer
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct