Apache Airflow Provider(s)

amazon

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==5.0.0

Apache Airflow version

2.4.4

Operating System

linux

Deployment

Docker-Compose

Deployment details

Custom built docker image based on the official one.

What happened

When I was migrating legacy EcsOperator to EcsRunTaskOperator I received this error:

airflow.exceptions.AirflowException: Invalid arguments were passed to EcsRunTaskOperator. Invalid arguments were:
**kwargs: {'aws_conn_id': 'aws_connection'}

From the source code and source code documentation it appears that aws_conn_id is a valid argument, but nevertheless the error gets thrown.

What you think should happen instead

EcsRunTaskOperator should work with provided aws_conn_id argument.

How to reproduce

Create an instance of EcsRunTaskOperator and provide valid aws_conn_id argument.

Anything else

During my investigation I compared current version of ecs module and previous one. From that investigation it's clear that aws_conn_id argument was removed from keyword arguments before it was passed to parent classes in the legacy version, but now it's not getting removed. In the end this error is caused by Airflow's BaseOperator receiving unknown argument aws_conn_id.

ecs.py L49

class EcsBaseOperator(BaseOperator):
    """This is the base operator for all Elastic Container Service operators."""

    def __init__(self, **kwargs):
        self.aws_conn_id = kwargs.get('aws_conn_id', DEFAULT_CONN_ID)
        self.region = kwargs.get('region')
        super().__init__(**kwargs)

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct