SSHRemoteJobOperator: inconsistent "remote_base_dir" validation #69813

Under which category would you file this issue?

Providers

Apache Airflow version

3.3

What happened and how to reproduce it?

(On a system I can't report from) I use SSHRemoteJobOperator with remote_base_dir="/tmp-data/airflow-ssh-jobs".

The operator starts and tracks the remote job till the end - as it has before the update of the provider.

But instead of finishing cleanly (as the remote command did after an hour...), the operator fails while cleaning up the job.
It raises a Value Error with the description:

Invalid job directory: '/tmp-data/airflow-ssh-jobs/af_ ....'. Expected Path under '/tmp/airflow-ssh-jobs' for safety.

Removing the remote_base_dir argument avoids this failure.

What you think should happen instead?

Either the remote_base_dir should be rejected when instantiating the operator or be accepted at the very end.

Looking at the code,

This is the validation used inside the cleanup

Operating System

Ubuntu 24.04

Deployment

Virtualenv installation

Apache Airflow Provider(s)

ssh

Versions of Apache Airflow Providers

python 3.12
apache-airflow-providers-ssh==5.0.3

Official Helm Chart version

Not Applicable

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

No response

Anything else?

Just occurred when updating from airflow 3.2.x to 3.3.0, incl apache-airflow-providers-ssh 5.0.3 ->5.0.2

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct