You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Current SFTPAdapter.openSession() uses a connection pool with limit: options?.concurrency ?? 1: concurrent transfers can create one SSH login per file in flight. The source even documents PERF-SFTP-MULTIPLEX as a future optimization: the ssh2 SFTP channel supports multiple concurrent request IDs, so a single SSH connection can potentially multiplex operations without N independent handshakes.
A maintained downstream currently uses one reusable SSH/SFTP client for concurrent source transfers, plus a bounded transient handshake retry. This reduces pressure on SFTP servers with strict concurrent-login limits, but may behave differently on certain SSH servers and is not automatically a drop-in replacement for the existing pool.
Proposed upstream design discussion
Consider an explicitly reviewable single-connection multiplex mode for SFTP source sessions, with the pool behavior preserved for compatibility until supported integration/performance evidence is available. The API/default choice should be made with maintainers, not silently changed by a fork.
Acceptance proposals:
A request for concurrency 8 with many files uses one SSH session in multiplex mode, while still having more than one file I/O operation in flight.
Connection lifecycle is fail-closed and race-safe: close exactly once, new requests after close or disconnect fail, mid-transfer disconnect is observable, and retry applies only to transient initial handshakes (not authentication).
Shared directory creation cache and concurrent file creation remain safe.
Existing pool mode retains current documented behavior and unit/integration expectations unless intentionally migrated.
Benchmarks compare multi-connection pooling and single-connection multiplexing on representative high-latency and restrictive SFTP servers, including large and many-small-file workloads.
No DB migrations, deployment-specific credentials or Homelab-only settings.
Source-tree directory scanning is separately covered by #186 / draft PR #187. This issue concerns file transfer sessions, not directory scans. Once an interface/default is accepted, an upstream-native PR can be prepared against the current dev baseline with test-first coverage.
changed the title [-]Consider opt-in single-connection SFTP multiplexing for concurrent file transfers[/-][+][Enhancement]: Consider opt-in single-connection SFTP multiplexing for concurrent file transfers[/+]on Oct 10, 2026
Thanks for this. It's already on my list, it's the PERF-SFTP-MULTIPLEX note in openSession(). The pool opens at most as many connections as Parallel transfers allows (4 by default, 8 at most) and reuses them for every file of the session, so this is about speed rather than something being broken. I'll treat it as an enhancement.
It changes how transfers are pooled, so I'd like to do it myself as its own change and measure it against a few different servers. Please hold off on a PR for this one.
A few things that will matter when I do:
The ssh2-sftp-client README advises against running requests in parallel over one connection, so it needs testing on more than OpenSSH.
Every request adds its own end, close and error listeners. With 12 requests in flight Node already prints a MaxListenersExceededWarning.
One connection also means one TCP connection and one SSH channel window for every file. Many small files should get faster, but a few large ones may get slower than over separate connections, so both need to be measured.
The walk from #187 is a good first data point. With a single connection it was just as fast as with 8.
Activity
[-]Consider opt-in single-connection SFTP multiplexing for concurrent file transfers[/-][+][Enhancement]: Consider opt-in single-connection SFTP multiplexing for concurrent file transfers[/+]Skyfay commentedon Oct 10, 2026
Hey @X1pheR
Thanks for this. It's already on my list, it's the
PERF-SFTP-MULTIPLEXnote inopenSession(). The pool opens at most as many connections as Parallel transfers allows (4 by default, 8 at most) and reuses them for every file of the session, so this is about speed rather than something being broken. I'll treat it as an enhancement.It changes how transfers are pooled, so I'd like to do it myself as its own change and measure it against a few different servers. Please hold off on a PR for this one.
A few things that will matter when I do:
ssh2-sftp-clientREADME advises against running requests in parallel over one connection, so it needs testing on more than OpenSSH.end,closeanderrorlisteners. With 12 requests in flight Node already prints aMaxListenersExceededWarning.The walk from #187 is a good first data point. With a single connection it was just as fast as with 8.
I'll keep this issue open to track it.
Thanks :)