Skip to content

Cannot retrieve config from alternative secrets backend. #26754

@Flogue

Description

@Flogue

Apache Airflow Provider(s)

amazon

Versions of Apache Airflow Providers

Providers included in apache/airflow:2.4.0 docker image:

apache-airflow==2.4.0
apache-airflow-providers-amazon==5.1.0
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.3.0
apache-airflow-providers-common-sql==1.2.0
apache-airflow-providers-docker==3.1.0
apache-airflow-providers-elasticsearch==4.2.0
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-google==8.3.0
apache-airflow-providers-grpc==3.0.0
apache-airflow-providers-hashicorp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-microsoft-azure==4.2.0
apache-airflow-providers-mysql==3.2.0
apache-airflow-providers-odbc==3.1.1
apache-airflow-providers-postgres==5.2.1
apache-airflow-providers-redis==3.0.0
apache-airflow-providers-sendgrid==3.0.0
apache-airflow-providers-sftp==4.0.0
apache-airflow-providers-slack==5.1.0
apache-airflow-providers-sqlite==3.2.1
apache-airflow-providers-ssh==3.1.0

Apache Airflow version

2.4

Operating System

AWS Fargate

Deployment

Docker-Compose

Deployment details

We have configure the alternative secrets backend to use AWS SMP:

[secrets]
backend = airflow.providers.amazon.aws.secrets.systems_manager.SystemsManagerParameterStoreBackend
backend_kwargs = {"config_prefix": "/airflow2/config", "connections_prefix": "/airflow2/connections", "variables_prefix": "/airflow2/variables"}

What happened

All processes fail with:

`Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/secrets/systems_manager.py", line 178, in _get_secret
    response = self.client.get_parameter(Name=ssm_path, WithDecryption=True)
  File "/home/airflow/.local/lib/python3.7/site-packages/cached_property.py", line 36, in __get__
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/secrets/systems_manager.py", line 98, in client
    from airflow.providers.amazon.aws.hooks.base_aws import SessionFactory
ImportError: cannot import name 'SessionFactory' from 'airflow.providers.amazon.aws.hooks.base_aws' (/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/configuration.py", line 122, in _get_config_value_from_secret_backend
    return secrets_client.get_config(config_key)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/secrets/systems_manager.py", line 167, in get_config
    return self._get_secret(self.config_prefix, key)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/secrets/systems_manager.py", line 180, in _get_secret
    except self.client.exceptions.ParameterNotFound:
  File "/home/airflow/.local/lib/python3.7/site-packages/cached_property.py", line 36, in __get__
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/secrets/systems_manager.py", line 98, in client
    from airflow.providers.amazon.aws.hooks.base_aws import SessionFactory
ImportError: cannot import name 'SessionFactory' from 'airflow.providers.amazon.aws.hooks.base_aws' (/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/secrets/systems_manager.py", line 178, in _get_secret
    response = self.client.get_parameter(Name=ssm_path, WithDecryption=True)
  File "/home/airflow/.local/lib/python3.7/site-packages/cached_property.py", line 36, in __get__
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/secrets/systems_manager.py", line 98, in client
    from airflow.providers.amazon.aws.hooks.base_aws import SessionFactory
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py", line 49, in <module>
    from airflow.models.connection import Connection
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/connection.py", line 32, in <module>
    from airflow.models.base import ID_LEN, Base
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/base.py", line 76, in <module>
    COLLATION_ARGS = get_id_collation_args()
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/base.py", line 70, in get_id_collation_args
    conn = conf.get('database', 'sql_alchemy_conn', fallback='')
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/configuration.py", line 557, in get
    option = self._get_option_from_secrets(deprecated_key, deprecated_section, key, section)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/configuration.py", line 577, in _get_option_from_secrets
    option = self._get_secret_option(section, key)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/configuration.py", line 502, in _get_secret_option
    return _get_config_value_from_secret_backend(secrets_path)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/configuration.py", line 125, in _get_config_value_from_secret_backend
    'Cannot retrieve config from alternative secrets backend. '
airflow.exceptions.AirflowConfigException: Cannot retrieve config from alternative secrets backend. Make sure it is configured properly and that the Backend is accessible.
cannot import name 'SessionFactory' from 'airflow.providers.amazon.aws.hooks.base_aws' (/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py)

`

What you think should happen instead

Airflow 2.3.4 was using amazon provider 5.0.0 and everything was working fine. Looking at the SystemsManagerParameterStoreBackend class, the client method changed in amazon 5.1.0 (coming with AF 2.4). There use to be a simple boto3.session call. The new code calls for an import of SessionFactory. I do not understand why this import fails though.

How to reproduce

I assume anyone that sets up the parameter store as backend and try to use the docker image (FROM apache/airflow:2.4.0) will run into this issue.

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions