-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Closed
Labels
area:providersgood first issuekind:featureFeature RequestsFeature Requestsprovider:googleGoogle (including GCP) related issuesGoogle (including GCP) related issues
Description
Description
hook = BigQueryHook('google_cloud_default', use_legacy_sql=False)
hook.get_sqlalchemy_engine()this one would raise this exception 👇
[2021-11-10 08:26:06,927] {taskinstance.py:1501} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1157, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1331, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1361, in _execute_task
result = task_copy.execute(context=context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/operators/python.py", line 150, in execute
return_value = self.execute_callable()
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/operators/python.py", line 161, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/opt/airflow/dags/dags/utils/others/subscription_related.py", line 98, in wrapper
return func(*args, **kwargs)
File "/opt/airflow/dags/dags/ods/ingest_emails/udfs/ingest_emails.py", line 59, in extract_and_load_campaign_emails
esp.extract_and_load_campaign_emails(context)
File "/opt/airflow/dags/dags/utils/factories/esp_factory/klaviyo_esp.py", line 230, in extract_and_load_campaign_emails
olap_session = engine2session(self.olap_hook.get_sqlalchemy_engine())
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/hooks/dbapi.py", line 100, in get_sqlalchemy_engine
return create_engine(self.get_uri(), **engine_kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/__init__.py", line 525, in create_engine
return strategy.create(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 61, in create
entrypoint = u._get_entrypoint()
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/url.py", line 172, in _get_entrypoint
cls = registry.load(name)
File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 277, in load
raise exc.NoSuchModuleError(
sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:google_cloud_platformUse case/motivation
I hope this sqlalchemy toy example works
from sqlalchemy.orm import sessionmaker, scoped_session
engine = hook.get_sqlalchemy_engine()
scoped_session(sessionmaker(bind=engine, autocommit=True))Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
area:providersgood first issuekind:featureFeature RequestsFeature Requestsprovider:googleGoogle (including GCP) related issuesGoogle (including GCP) related issues