Apache Airflow Provider(s)

amazon

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==4.1.0

Apache Airflow version

2.2.2

Operating System

Amazon Linux 2

Deployment

MWAA

Deployment details

No response

What happened

The current state of the SQSPublishOperator does not support FIFO queues because:
From the Boto3 documentation, a FIFO queue requires a MessageGroupId parameter -> SQS.Client.send_message
The SQSHook calls the boto3 client with just these parameters:

.send_message(
            QueueUrl=queue_url,
            MessageBody=message_body,
            DelaySeconds=delay_seconds,
            MessageAttributes=message_attributes or {},

        )

so if queue_url happens to be a FIFO queue, then that API call will fail and therefore the Airflow Operator will fail too.

What you think should happen instead

The SQSPublishOperator should support FIFO queues by accepting the MessageGroupId as a parameter.

How to reproduce

1. Setup a FIFO SQS queue
2. Setup an Airflow operator:

task1 = SQSPublishOperator(
task_id = task1,
sqs_queue = "url",
message_content = "Message string",
delay_seconds = 0,
)

Anything else

N/A

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct