Add Async Support to Notifiers

Body

Some of the projects that are in flight would benefit from having the existing Notifiers support async usage. The SlackWebhookNotifier and SMPT Notifier have been done already and can be used as examples, and we are looking to mob-code the rest quickly.

Amazon

Completed	Notifier Name	Priority	Owner	PR Link
❌	ChimeNotifier	See note [1] below	NONE	NONE
✅	SnsNotifier		ferruzzi	#56133
✅	SqsNotifier		ferruzzi	#56159
✅	SesNotifier		ferruzzi	#56106

Apprise

Completed	Notifier Name	Priority	Owner	PR Link
✅	AppriseNotifier		dondaum	#57541

Atlassian Jira

Completed	Notifier Name	Priority	Owner	PR Link
✅	JiraNotifier	HIGH	dondaum	#56326

Discord

Completed	Notifier Name	Priority	Owner	PR Link
✅	DiscordNotifier		dondaum	#56911

HTTP

Completed	Notifier Name	Priority	Owner	PR Link
✅	HttpNotifier		seanghaeli	#56160

Opsgenie

Completed	Notifier Name	Priority	Owner	PR Link
❌	OpsgenieNotifier	See note [2] below

Pagerduty

Completed	Notifier Name	Priority	Owner	PR Link
✅	PagerdutyNotifier	HIGH	dondaum	#56540

Slack

Completed	Notifier Name	Priority	Owner	PR Link
✅	SlackNotifier	See thread below	dondaum	#56685
✅	SlackWebhookNotifier	HIGH	ramitkataria	#53831

Simple Mail Transfer Protocol (SMTP)

Completed	Notifier Name	Priority	Owner	PR Link
✅	SmtpNotifier	HIGH	ferruzzi	#55308

NOTE 1: Amazon announced they are shutting Chime down, no real point adding this one
Note 2: Atlassian has announced an end to OpsGenie here, we're going to skip this one

Committer

I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Hello, I would like to contribute here.

I checked the Python libraries used for JiraNotifier and PagerdutyNotifier as they are ranked HIGH priority.

It seems they don't support Asyncio yet. I also looked for asynchronous alternatives but couldn't find anything suitable.

We could use asgiref sync_to_async, which we already use internally as a trigger for calling sync methods in asynchronous logic.

Or I can start with other notfiers.

Any ideas or thoughts on this?

@seanghaeli

Thanks for looking into them, that is disappointing. I know we had to go with a 3rd party async library for the smtp one and had hoped there would be options for the others, but I suppose that makes sense since SMTP is a pretty universal standard and not a proprietary protocol. I picked them as high priority because they "felt" like ones that folks would use more often, but I don't have any data to back up that gut feeling.

One option that was mentioned elsewhere for another Notifier was making use of a generic HTTP notifier if either of those have an HTTP API that could be used? I think @seanghaeli is planning to work on an HTTP notifier, but maybe if you can get to it first and if it can be used to those two, that could be an option.

I'm still pretty new to async stuff in python, which is why I wanted to work on this and learn, so I can't really say offhand if asgiref sync_to_asyn is the right call or not. If you are familiar with it and think it'll work, and we don't have the option of using an API with an async http call then sure, give it a shot.

I was going to start on a new AWS SES Notifier (both sync and async versions) in the next couple days by request, so if none of the above sounds good to you, you can claim any of the others that are listed.

@ramitkataria

Looking at it more, and after a chat with @ramitkataria , we can see three approaches for JIRA, and option three is the clear winner:

Wrapping the notifier in the syc_to_async method will block the Triggerer and will likely work but isn't ideal. Let's leave that as a last resort.
We could put in a PR to https://github.com/atlassian-api/atlassian-python-api/tree/master/atlassian to add async support. That adds a dependency to their project which it's up to them to maintain, plus there will then be a wait for approval/merge/release/etc on their side, THEN a wait for Airflow to ingest that and bump our Jira minimum version so it's a LOOONG term answer which depends on them accepting and maintaining the work. I think we should avoid this option, even if it's maybe the "right" answer, it's relying heavily on others.
We can make a JiraAsyncHook which inherits HttpAsyncHook and has a create_issue method to assemble the URL and call post using the aiohttp connection from HttpAsyncHook to make the call. Looking at their code, most of what they are doing is building the URL and then calling request(POST) which we can replicate pretty easily with minimal code.

I think Pagerduty looks like a similar solution should work too, but I haven't looked as closely there.

Thank you for taking the time to develop these different options. I agree, and Option 3 looks promising. I'll try implementing an asynchronous version of JiraNotifier based on Option 3.

NOTE 2: SlackNotifier uses a deprecated API we shouldn't encourage new use

Can you reference slack docs for it? I don't see we deprecated the notifier so I am a bit confused by this statement.

Slack has deprecated that token-based endpoint on their end and wants users to use the Webhook endpoint

I may have been mistaken, there are some deprecated "legacy token" types but I may have misunderstood what I was reading before, or maybe I stumbled on an old doc page on accident. This page seems to imply that tokens are still a thing, but then explains how to use the token with the Webhook endpoint and doesn't mention the WebClient. I'll remove that note and someone can look into it more.

I have opened a PR for the JiraNotifier I can take over PagerdutyNotifier next.

I have opened a PR for the PagerdutyNotifier I can take over SlackNotifier next.

Knocking these out quick! Thanks. I'll update the checklist

I have opened a PR for the SlackNotifier I can take over DiscordNotifier next.

Added Sean's HTTP Notifier PR to the table, for the sake of completion

I'm wondering if we should exclude OpsgenieNotifier.

I checked the Python library we use and it hasn't been updated in 4 years. Link

So I did some research and found out that Atlassian is shutting down Opsgenie. There is also a guide for it
to migrate from there to standard Atlassian products: https://www.atlassian.com/software/opsgenie/migration

In my opinion, it may not be worth the effort. WDYT?

I have opened a PR for the DiscordNotifier I can take over AppriseNotifier next.

I think we can skip OpsGenie then. It's still a couple years out on full shutdown, but if they are steering users away form it, I think we're safe not encouraging new uses. I'll mark it in the chart.

I have opened a PR for the AppriseNotifier.

I think we're done here, right? If not, or if there is anything else I can help with, just drop me a message.

Yeah! Looks like we're done here. Thanks a ton!

Navigation Menu

Repository navigation

Add Async Support to Notifiers #55237

Body

Committer

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add Async Support to Notifiers #55237

Description

Body

Committer

Activity

dondaum commented on Sep 17, 2025

ferruzzi commented on Sep 17, 2025

ferruzzi commented on Sep 18, 2025

dondaum commented on Sep 18, 2025

eladkal commented on Oct 1, 2025

ferruzzi commented on Oct 1, 2025

ferruzzi commented on Oct 3, 2025

dondaum commented on Oct 4, 2025

dondaum commented on Oct 10, 2025

ferruzzi commented on Oct 14, 2025

dondaum commented on Oct 16, 2025

ferruzzi commented on Oct 20, 2025

dondaum commented on Oct 21, 2025

dondaum commented on Oct 21, 2025

ferruzzi commented on Oct 21, 2025

dondaum commented on Nov 5, 2025

dondaum commented on Nov 20, 2025

ferruzzi commented on Nov 25, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions