Apache Airflow version

2.2.4 (latest released)

What happened

I am trying to create an SFTP connection which includes the extra params of private_key which includes a txt output of my private key. Ie: {"look_for_keys": "false", "no_host_key_check": "true", "private_key": "-----BEGIN OPENSSH PRIVATE KEY----- keygoeshere==\n----END OPENSSH PRIVATE KEY-----"}

When I test the connection I get the error expected str, bytes or os.PathLike object, not Ed25519Key

When I try and use this connection I get the following error:

  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/sftp/hooks/sftp.py", line 208, in list_directory
    conn = self.get_conn()
  File "/home/airflow/.local/lib/python3.7/site-packages/tenacity/__init__.py", line 324, in wrapped_f
    return self(f, *args, **kw)
  File "/home/airflow/.local/lib/python3.7/site-packages/tenacity/__init__.py", line 404, in __call__
    do = self.iter(retry_state=retry_state)
  File "/home/airflow/.local/lib/python3.7/site-packages/tenacity/__init__.py", line 349, in iter
    return fut.result()
  File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 428, in result
    return self.__get_result()
  File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/home/airflow/.local/lib/python3.7/site-packages/tenacity/__init__.py", line 407, in __call__
    result = fn(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/sftp/hooks/sftp.py", line 172, in get_conn
    self.conn = pysftp.Connection(**conn_params)
  File "/home/airflow/.local/lib/python3.7/site-packages/pysftp/__init__.py", line 142, in __init__
    self._set_authentication(password, private_key, private_key_pass)
  File "/home/airflow/.local/lib/python3.7/site-packages/pysftp/__init__.py", line 164, in _set_authentication
    private_key_file = os.path.expanduser(private_key)
  File "/usr/local/lib/python3.7/posixpath.py", line 235, in expanduser
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not Ed25519Key

This only seems to happen for Ed25519 keys. RSA worked fine!

What you think should happen instead

It should work, I don't specify this as an Ed25519Key I think the connection manager code is saving it as a paraminko key but when testing / using it as a DAG it is expecting a string!

I don't see why you can't save it as a paraminko key and use it in the connection.

Also it seems to work fine when using RSA keys, but super short keys are cooler!

How to reproduce

Create a new Ed25519 ssh key and a new SFTP connection and copy the following into the extra field:

{"look_for_keys": "false", "no_host_key_check": "true", "private_key": "-----BEGIN RSA PRIVATE KEY----- Ed25519_key_goes_here -----END RSA PRIVATE KEY-----"}

Test should yield the failure TypeError: expected str, bytes or os.PathLike object, not Ed25519Key

Operating System

RHEL 7.9 on host OS and Docker image for the rest.

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==3.0.0
apache-airflow-providers-celery==2.1.0
apache-airflow-providers-cncf-kubernetes==3.0.2
apache-airflow-providers-docker==2.4.1
apache-airflow-providers-elasticsearch==2.2.0
apache-airflow-providers-ftp==2.0.1
apache-airflow-providers-google==6.4.0
apache-airflow-providers-grpc==2.0.1
apache-airflow-providers-hashicorp==2.1.1
apache-airflow-providers-http==2.0.3
apache-airflow-providers-imap==2.2.0
apache-airflow-providers-microsoft-azure==3.6.0
apache-airflow-providers-mysql==2.2.0
apache-airflow-providers-odbc==2.0.1
apache-airflow-providers-postgres==3.0.0
apache-airflow-providers-redis==2.0.1
apache-airflow-providers-sendgrid==2.0.1
apache-airflow-providers-sftp==2.4.1
apache-airflow-providers-slack==4.2.0
apache-airflow-providers-sqlite==2.1.0
apache-airflow-providers-ssh==2.4.0

Deployment

Other Docker-based deployment

Deployment details

Docker image of 2.2.4 release with VERY minimal changes. (wget, curl, etc added)

Anything else

RSA seems to work fine... only after a few hours of troubleshooting and writing this ticket did I learn that. 😿

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct