You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently scopes for google libraries are taken from connection parameters, this PR creates an additional parameter impersonation_scopes that will be used in situations when impersonation_chain is being used.
The flow to get credentials for gcp providers goes something like this
A library like bigquery will call get_credentials()
I need some help figuring out how to write a test. I see that there is a cloud/operators/test_bigquery.py file with a impersonation_chain call but no impersonation_chain call in cloud/hooks/test_bigquery.py
I need some help figuring out how to write a test. I see that there is a cloud/operators/test_bigquery.py file with a impersonation_chain call but no impersonation_chain call in cloud/hooks/test_bigquery.py
You could mock the impersonation chain as done in other tests of GCP hooks (hooks/test_compute.py, for example; there are more).
As for the other example, it is called google_impersonation_chain to differentiate it from a similar parameter in the transfer operator's target/source cloud platform (which is not necessarily GCP).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes: #33400
Currently
scopesfor google libraries are taken from connection parameters, this PR creates an additional parameterimpersonation_scopesthat will be used in situations whenimpersonation_chainis being used.The flow to get credentials for gcp providers goes something like this
airflow/airflow/providers/google/cloud/hooks/bigquery.py
Line 163 in 00cd44e
airflow/airflow/providers/google/common/hooks/base_google.py
Line 307 in 00cd44e
self.scopes()airflow/airflow/providers/google/common/hooks/base_google.py
Line 290 in 00cd44e
_get_scopes())airflow/airflow/providers/google/common/hooks/base_google.py
Line 414 in 00cd44e
Possible alternative solutions
scopes()definition in custom hook - as detailed in above issue BigQuery with impersonation_chain does not accept custom scopes #33400 (comment)scopes- it would be confusing what takes precedence if there is scopes in parameter and connectionI would need some help testing and bringing this over the finish line
Should be able to do something like