Skip to content

Kubernetes - related backports needs some fixes for xcom cases #11298

@potiuk

Description

@potiuk

The "cncf.kubernetes" still has problems when releasing backport packages.

Reported by @mik-laj in https://lists.apache.org/thread.html/rd7b8fa87ac8117812e9883b342876eaee6039ce8e421cab3e454406c%40%3Cdev.airflow.apache.org%3E:

Apache Airflow version:

Master Backport packages for "cncf.kubernetes" and "google"
Installed on top of 1.10.12

What happened:

The example_spark_kubernetes_spark_pi.yaml from our repository and example DAG works

  • airflow/providers/cncf/kubernetes/example_dags/example_kubernetes.py,
  • airflow/providers/cncf/kubernetes/example_dags/example_spark_kubernetes_spark_pi.yaml

But it does not work with example_kubernetes_operator. I performed the following steps which ended with an error.

$ curl -LO https://raw.githubusercontent.com/apache/airflow/master/airflow/providers/cncf/kubernetes/example_dags/example_kubernetes.py
$ cat example_kubernetes.py | sed 's/operators.bash/operators.bash_operator/g' | sed 's/in_cluster=True/in_cluster=False/g' > a.py
$ mv a.py example_kubernetes.py
$ airflow test example_kubernetes_operator write-xcom 2010-01-01 
.....
[2020-10-04 23:29:06,623] {taskinstance.py:1150} ERROR - type object 'PodGenerator' has no attribute 'add_xcom_sidecar'
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 984, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 269, in execute
    self.pod = self.create_pod_request_obj()
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 405, in create_pod_request_obj
    pod = PodGenerator.add_xcom_sidecar(pod)
AttributeError: type object 'PodGenerator' has no attribute 'add_xcom_sidecar'

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions