Description

I think we are really close to release the second wave of Backport Packages. We accumulated a number of changes already and few more are about to be completed so that we are, I think, close to release the second wave.

** What will be in those packages **

Everything that is currently merged to the master branch in "providers" package will be included.

Those are open issues that we know we should merge before though:

• Enabling cncf.kubernetes provider after 1.10.11/1.10.12 backported changes and spark-on-kubernetes operator - this is deferred to the next backport release as per Kubernetes - related backports needs some fixes for xcom cases #11298
• Implementing Impersonation for all applicable Google operators (@olchas works on it with Hooks already merged)
• DataProc operator improvements (@turbaszek works on it in Improve handling Dataproc cluster creation with ERROR state #9593 )
• Improve idempotency of BigQueryInsertJobOperator (@turbaszek works on it in Improve idempotency of BigQueryInsertJobOperator #9590)
• Release 1.10.12 of Airflow as it fixes some teething problems with Kubernetes refactor from 1.10.11
• Fix BigQuery operators problem with serialization BIgQuery operators from backport packages and DAG Serialization #10171
• AWS SES Add Amazon SES hook #10004
• Fix the DLP operators.
• finalize and implement the POC/WIP on provider package discovery mechanism #10822 allowing to release 1.10.13 with support to auto-dicover provider capabilities such as connections and extra links deferred to another release
• Merge Add Kerberos Auth for Presto #10488 with Kerberos on Presto change. deferred to the next release
• Breaking changes: Email operator moved to Core: we are going to address it in 1.10.13 by adding email provider there and releasing provider package via Core operators/hooks import paths according to AIP-21 #11178

We have to make the following adjustments to the process:

• make sure that we only release the packages that have some changes #back Make sure Only changed backports get released #10375
• make sure to add changes from the previous releases - not only commits but add mechanism to add
  backwards-incompatible changes akin to UPDATING.md Add mechanism to maintain UPDATING.md for each provider package separately #10376

Update:
The second wave is delayed a bit (but coming soon). The reason is two-fold

• the cncf.kubernetes package has a problem that we are solving
• we might try to get the mechanism of separate providers packages and "discovery" mechanism before we build the packages - see the POC/WIP on provider package discovery mechanism #10822 and POC/WIP on provider package discovery mechanism (1.10) #10823

Use case / motivation

We want the users of airflow 1.10 to benefit from fixes and improvements in the latest packages.