Problem
FileSyncManager uploads ~581 files (skills, credentials, caches) on sandbox init. Sequential per-file uploads are painfully slow on every remote backend.
Benchmarked on Daytona (581 files):
| Method |
Time |
Sequential (upload_file × 581) |
803s (13.4 min) |
Bulk (upload_files × 1) |
4.3s |
Daytona is fixed in #7447 using the SDK's upload_files() multipart POST. SSH and Modal still suffer from the same sequential bottleneck.
Sub-issues
Context
FileSyncManager now supports an optional bulk_upload_fn callback (#7447). When provided, all changed files go through it in one call instead of iterating per-file. Backends just need to provide the callback.
Backend status
| Backend |
Upload method |
Bulk support |
Bottleneck |
| Daytona |
SDK upload_files() |
✔ #7447 |
— |
| SSH |
scp per file over ControlMaster |
✗ |
Per-file subprocess + round-trip |
| Modal |
base64 | exec per file |
✗ |
Per-file exec + encoding overhead |
| Docker |
Bind mount |
N/A |
No sync needed |
| Singularity |
Bind mount |
N/A |
No sync needed |
Related
Problem
FileSyncManageruploads ~581 files (skills, credentials, caches) on sandbox init. Sequential per-file uploads are painfully slow on every remote backend.Benchmarked on Daytona (581 files):
upload_file× 581)upload_files× 1)Daytona is fixed in #7447 using the SDK's
upload_files()multipart POST. SSH and Modal still suffer from the same sequential bottleneck.Sub-issues
tar czf | ssh tar xzforrsync --files-frombase64 | execper file. No SDK bulk upload API exists. Options:execcall (tar/base64 archive)Context
FileSyncManagernow supports an optionalbulk_upload_fncallback (#7447). When provided, all changed files go through it in one call instead of iterating per-file. Backends just need to provide the callback.Backend status
upload_files()scpper file over ControlMasterbase64 | execper fileRelated
bulk_upload_fntoFileSyncManager+ Daytona wiring