Skip to content

feat-51094: add OAuth 2 / XOAUTH2 support via auth_type & token/credential extras #53554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
eladkal merged 15 commits into from
Aug 5, 2025
Merged

feat-51094: add OAuth 2 / XOAUTH2 support via auth_type & token/credential extras #53554

eladkal merged 15 commits into from
Aug 5, 2025

Conversation

@nailo2c
Copy link
Contributor

@nailo2c nailo2c commented Jul 19, 2025

Closes: #51094

Why

Airflow does not currently support OAuth for SMTP. Microsoft is removing BasicAuth as capability from Exchange Online globally this September 2025 (Source: Microsoft announcement).

How

Area Change
Core send_email_smtp / send_mime_email now accept auth_type="oauth2" and an optional access_token.
Provider SmtpHook, SmtpNotifier, and EmailOperator accept auth_type; the hook reads either access_token or client_id + client_secret from Connection extra.
UI Connection form exposes the new fields (auth_type, access_token, client_id, …).
Docs Updated connections/smtp.rst.
Tests New unit tests cover both Basic Auth and OAuth 2 paths.

What

Step 1

Use the Goole Oauth Playground to obtain an access_token.

Step 2

Start Airflow locally with breeze start-airflow and add a connection:

airflow connections add 'smtp_gmail_oauth2' \
  --conn-type smtp \
  --conn-host smtp.gmail.com \
  --conn-port 587 \
  --conn-login '[email protected]' \
  --conn-extra '{
    "from_email": "[email protected]",
    "auth_type": "oauth2",
    "access_token": "ya29.*****",
    "disable_ssl": "true"
  }'

Step 3

Tested by run this DAG

from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.providers.smtp.hooks.smtp import SmtpHook
from datetime import datetime

def gmail_oauth2_test():
    with SmtpHook(smtp_conn_id="smtp_gmail_oauth2", auth_type="oauth2") as hook:
        hook.send_email_smtp(
            to="[email protected]",
            subject="[Airflow→Gmail] OAuth2 OK",
            html_content="<h3>Gmail XOAUTH2 works 🎉</h3>",
        )

with DAG(
    dag_id="test_gmail_oauth2",
    start_date=datetime(2025,7,1),
    schedule=None,
    catchup=False,
) as dag:
    PythonOperator(task_id="send_mail", python_callable=gmail_oauth2_test)

Or add in airflow.cfg

[smtp]
smtp_host = smtp.gmail.com
smtp_port = 587
smtp_starttls = True
smtp_ssl = False

Then in a Python shell:

from airflow.utils.email import send_email_smtp
send_email_smtp(
    to="[email protected]",
    subject="[Airflow utils] XOAUTH2 test",
    html_content="<h3>🎉 utils.send_email_smtp works</h3>",
    conn_id="smtp_gmail_oauth2",
    auth_type="oauth2",
    access_token="ya29.*****"
)

Step 4

Confirm the message arrived in mailbox.
截圖 2025-07-19 下午2 53 06

@nailo2c nailo2c requested a review from hussein-awala as a code owner July 19, 2025 22:02
@nailo2c nailo2c closed this Jul 20, 2025
@nailo2c nailo2c reopened this Jul 20, 2025
eladkal
eladkal reviewed Jul 20, 2025
View reviewed changes
@nailo2c
Copy link
Contributor Author

nailo2c commented Jul 28, 2025

Hi @eladkal, I’ve moved all the changes from email.py into SmtpHook.

With that done, I think this PR is ready to merge for the new feature, and we can handle deprecating email.py in a follow-up PR. Let me know what you think. Thanks for your guidance!

@eladkal
Copy link
Contributor

eladkal commented Jul 28, 2025

Nice!
I think what we are missing is some doc explnations.
Right now there are many settings to configure but it's not easy to know which setting relates to what.
I think we need to have something like the AWS docs:
https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/connections/aws.html

where we explain the different options and how to set them.

we can handle deprecating email.py in a follow-up PR.

That would be great!

@potiuk
Copy link
Member

potiuk commented Jul 29, 2025

Right now there are many settings to configure but it's not easy to know which setting relates to what.
I think we need to have something like the AWS docs:
https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/connections/aws.html

Agree. More user docs is better.

@nailo2c
Copy link
Contributor Author

nailo2c commented Jul 29, 2025

Sure, let me work on that.

@nailo2c
Copy link
Contributor Author

nailo2c commented Aug 4, 2025

Done. I referred to aws.rst to make the document more organized and to add clearer explanations. I also provided additional runnable examples wherever possible.

截圖 2025-08-03 下午6 16 48

@eladkal eladkal merged commit 42f581f into apache:main Aug 5, 2025
71 checks passed
Nataneljpwd pushed a commit to Asquator/airflow that referenced this pull request Aug 5, 2025
@nailo2c
Add OAuth 2 / XOAUTH2 support via auth_type & token/credential extr…
6e7a415 
…as (apache#53554)

* support auth_type=oauth2 for smtp

* add auth_type to notifications

* add ut for build_xoauth2_string & send_mime_email

* add test for smtp

* add test for notifications

* add ui widgets & property for oauth2

* add extra parameters to the docs

* fix CI test

* fix CI test apache#2

* fix CI test apache#3

* expose `build_xoauth2_string` & add fallback for old cores

* rollback email.py and test_email.py & refactor smtp.py

* refactor the doc to enrich examples and explnations
HsiuChuanHsu pushed a commit to HsiuChuanHsu/airflow that referenced this pull request Aug 5, 2025
@nailo2c @HsiuChuanHsu
Add OAuth 2 / XOAUTH2 support via auth_type & token/credential extr…
d2bc7d9 
…as (apache#53554)

* support auth_type=oauth2 for smtp

* add auth_type to notifications

* add ut for build_xoauth2_string & send_mime_email

* add test for smtp

* add test for notifications

* add ui widgets & property for oauth2

* add extra parameters to the docs

* fix CI test

* fix CI test apache#2

* fix CI test apache#3

* expose `build_xoauth2_string` & add fallback for old cores

* rollback email.py and test_email.py & refactor smtp.py

* refactor the doc to enrich examples and explnations
@eladkal eladkal mentioned this pull request Aug 7, 2025
Closed
40 tasks
ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 7, 2025
@nailo2c @ferruzzi
Add OAuth 2 / XOAUTH2 support via auth_type & token/credential extr…
bea2088 
…as (apache#53554)

* support auth_type=oauth2 for smtp

* add auth_type to notifications

* add ut for build_xoauth2_string & send_mime_email

* add test for smtp

* add test for notifications

* add ui widgets & property for oauth2

* add extra parameters to the docs

* fix CI test

* fix CI test #2

* fix CI test #3

* expose `build_xoauth2_string` & add fallback for old cores

* rollback email.py and test_email.py & refactor smtp.py

* refactor the doc to enrich examples and explnations
fweilun pushed a commit to fweilun/airflow that referenced this pull request Aug 11, 2025
@nailo2c @fweilun
Add OAuth 2 / XOAUTH2 support via auth_type & token/credential extr…
8cc09e3 
…as (apache#53554)

* support auth_type=oauth2 for smtp

* add auth_type to notifications

* add ut for build_xoauth2_string & send_mime_email

* add test for smtp

* add test for notifications

* add ui widgets & property for oauth2

* add extra parameters to the docs

* fix CI test

* fix CI test apache#2

* fix CI test apache#3

* expose `build_xoauth2_string` & add fallback for old cores

* rollback email.py and test_email.py & refactor smtp.py

* refactor the doc to enrich examples and explnations
@boelaars
Copy link

boelaars commented Aug 25, 2025

@eladkal Do you know if this is intended to make its way into the 2.x Airflow release at some point, or will it only be 3.x ? (Hope it's alright to ask here.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eladkal eladkal eladkal approved these changes

@hussein-awala hussein-awala Awaiting requested review from hussein-awala hussein-awala is a code owner

Assignees

No one assigned

Labels

area:providers kind:documentation provider:smtp

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

OAuth for SMTP notifications

4 participants

@nailo2c @eladkal @potiuk @boelaars