Importing entry_point plugins fails on Airflow v1.10.14 and Python 3.8 #13063
Description
Apache Airflow version: 1.10.14
Kubernetes version (if you are using kubernetes) (use kubectl version): N/A
Environment:
- Cloud provider or hardware configuration:
- OS (e.g. from /etc/os-release): Debian GNU/Linux 10 (buster)
- Kernel (e.g.
uname -a): Linux 3f9779a91397 5.4.0-52-generic I need a flow to start with a sensor #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020 x86_64 GNU/Linux - Others: Python 3.8
What happened:
On Airflow 1.10.14 and Python 3.8 importing plugins that have been provided through a setuptools entrypoint fails with the following error:
[2020-12-14 13:35:12,200] {{plugins_manager.py:159}} ERROR - Failed to import plugin gantry
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 150, in load_entrypoint_plugins
plugin_obj.__usable_import_name = entry_point.module
AttributeError: 'EntryPoint' object has no attribute 'module'
/usr/local/lib/python3.8/site-packages/gantry_plugin/plugin.py:11: FutureWarning: Registering operators or sensors in plugins is deprecated -- these should be treated like 'plain' python modules, and imported normally in DAGs.
Airflow 2.0 has removed the ability to register these types in plugins. See <http://airflow.apache.org/docs/stable/howto/custom-operator.html>.
class GantryPlugin(AirflowPlugin):
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 150, in load_entrypoint_plugins
plugin_obj.__usable_import_name = entry_point.module
AttributeError: 'EntryPoint' object has no attribute 'module'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 25, in <module>
from airflow.configuration import conf
File "/usr/local/lib/python3.8/site-packages/airflow/__init__.py", line 100, in <module>
operators._integrate_plugins()
File "/usr/local/lib/python3.8/site-packages/airflow/operators/__init__.py", line 107, in _integrate_plugins
from airflow.plugins_manager import operators_modules
File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 229, in <module>
plugins = load_entrypoint_plugins(
File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 160, in load_entrypoint_plugins
import_errors[entry_point.module] = str(e)
AttributeError: 'EntryPoint' object has no attribute 'module'
This plugin can be imported without issues on Airflow v1.10.12 and v1.10.13.
What you expected to happen:
I had expected Airflow to import the plugin without issues.
How to reproduce it:
I believe one should be able to reproduce this by creating a plugin that uses setuptools' entry_points mechanism as described here.
Anything else we need to know:
I believe this bug may be related to the changes introduced in b49838f. This issue also manifests itself despite of the efforts in #12859.