Skip to content

Commit 4b2f61f

Browse files
committed
feat(cli): replace rsync with tar-over-SSH for sandbox file sync (!41)
Closes #20, #36, #46 ## Summary - Replace the external `rsync` dependency with built-in tar-over-SSH for file synchronization, using the `tar` crate to stream archives through the existing SSH proxy tunnel - Add a new `nav sandbox sync` subcommand supporting bidirectional file transfer (`--up` to push local files, `--down` to pull sandbox files) - Add sync workflow example in `examples/sync-files.md` ## Changes | File | Change | |---|---| | `crates/navigator-cli/Cargo.toml` | Add `tar = "0.4"` dependency | | `crates/navigator-cli/src/ssh.rs` | Remove `sandbox_rsync()`, add `sandbox_sync_up()`, `sandbox_sync_down()`, `sandbox_sync_up_files()` | | `crates/navigator-cli/src/run.rs` | Add `sandbox_sync_command()` dispatcher, update re-exports, update `--sync` call site | | `crates/navigator-cli/src/main.rs` | Add `Sync` variant to `SandboxCommands` with `--up`/`--down` flags | | `architecture/sandbox-connect.md` | Rewrite File Sync section for tar-over-SSH and new command | | `examples/sync-files.md` | New example walkthrough for sync workflows | ## New UX ```bash # Push local files up to sandbox nav sandbox sync my-sandbox --up ./src /sandbox/src # Pull sandbox files down to local nav sandbox sync my-sandbox --down /sandbox/output ./output # Existing --sync flag on create still works (now uses tar internally) nav sandbox create --sync -- python main.py ``` ## Design - **No new host dependencies**: `tar` crate handles archive creation/extraction in Rust; `ssh` is already required. The sandbox side uses the `tar` binary from the base image. - **No new server-side infrastructure**: reuses the existing SSH tunnel, no new gRPC RPCs or gateway changes. - **Bidirectional**: the same tar-pipe pattern works for both push and pull by reversing which side produces and which consumes the archive. - **No compression** for v1 — the SSH tunnel is local-network; can add gzip via `flate2` later. ## Test Plan - All existing tests pass (`mise run pre-commit` green: fmt, clippy, 206 Rust tests, 6 Python tests) - Manual e2e testing required for actual sandbox sync operations
1 parent 8c01a83 commit 4b2f61f

10 files changed

Lines changed: 897 additions & 53 deletions

File tree

‎Cargo.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎architecture/sandbox-connect.md‎

Lines changed: 38 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -244,18 +244,47 @@ If `timeout_seconds > 0`, the exec is wrapped in `tokio::time::timeout`. On time
244244

245245
### File Sync
246246

247-
When `--sync` is passed to `sandbox create`, the CLI rsyncs local files into the sandbox after it reaches `Ready` and before any command runs.
247+
File sync uses **tar-over-SSH**: the CLI streams a tar archive through the existing SSH proxy tunnel. No external dependencies (like `rsync`) are required on the client side. The sandbox image provides GNU `tar` for extraction.
248248

249-
**File**: `crates/navigator-cli/src/ssh.rs` -- `sandbox_rsync()`
249+
**Files**: `crates/navigator-cli/src/ssh.rs`, `crates/navigator-cli/src/run.rs`
250250

251-
1. `crates/navigator-cli/src/run.rs` -- `git_repo_root()` determines the repository root via `git rev-parse --show-toplevel`
251+
#### `sandbox create --sync`
252+
253+
When `--sync` is passed to `sandbox create`, the CLI pushes local git-tracked files into `/sandbox` after the sandbox reaches `Ready` and before any command runs.
254+
255+
1. `git_repo_root()` determines the repository root via `git rev-parse --show-toplevel`
252256
2. `git_sync_files()` lists files with `git ls-files -co --exclude-standard -z` (tracked + untracked, respecting gitignore, null-delimited)
253-
3. `sandbox_rsync()` creates a new SSH session config (same as connect), then invokes:
254-
```
255-
rsync -az --from0 --files-from=- --relative -e "ssh -o ProxyCommand=... -o StrictHostKeyChecking=no ..." . sandbox:/sandbox
256-
```
257-
4. File paths are written to rsync's stdin as null-delimited entries
258-
5. Files land in `/sandbox` inside the container
257+
3. `sandbox_sync_up_files()` creates an SSH session config, spawns `ssh <proxy> sandbox "tar xf - -C /sandbox"`, and streams a tar archive of the file list to the SSH child's stdin using the `tar` crate
258+
4. Files land in `/sandbox` inside the container
259+
260+
#### `nav sandbox sync` command
261+
262+
The standalone `sandbox sync` subcommand supports bidirectional file transfer:
263+
264+
```bash
265+
# Push local files up to sandbox
266+
nav sandbox sync <name> --up <local-path> [<sandbox-path>]
267+
268+
# Pull sandbox files down to local
269+
nav sandbox sync <name> --down <sandbox-path> [<local-path>]
270+
```
271+
272+
- **Push (`--up`)**: `sandbox_sync_up()` streams a tar archive of the local path to `ssh ... tar xf - -C <dest>` on the sandbox side. Default destination: `/sandbox`.
273+
- **Pull (`--down`)**: `sandbox_sync_down()` runs `ssh ... tar cf - -C <dir> <path>` on the sandbox side and extracts the output locally via `tar::Archive`. Default destination: `.` (current directory).
274+
- No compression for v1 — the SSH tunnel is local-network; compression adds CPU cost with marginal bandwidth savings.
275+
276+
#### Why tar-over-SSH instead of rsync
277+
278+
| | tar-over-SSH | rsync |
279+
|---|---|---|
280+
| **Client dependency** | None — `tar` crate is compiled into the CLI | Requires `rsync` installed on the client machine |
281+
| **Sandbox dependency** | GNU `tar` (present in every base image) | Requires `rsync` installed in the container |
282+
| **Bidirectional** | Same pipe pattern reversed for push/pull | Needs different invocation or rsync daemon for pull |
283+
| **Transport complexity** | Single process (`ssh ... tar xf -`) | Two processes coordinating a delta-transfer protocol through the proxy tunnel |
284+
| **Incremental sync** | Re-sends everything every time | Only transfers changed blocks (faster for repeated syncs of large repos) |
285+
| **Compression** | Uncompressed (can add gzip via `flate2` later) | Built-in `-z` flag |
286+
287+
For Navigator's use case — one-shot or on-demand pushes of project files over a local network tunnel — the incremental sync advantage of rsync is marginal. Eliminating the external dependency and getting clean bidirectional support outweigh the delta-transfer benefit. If repeated rapid re-syncs of large repos become a need (e.g., a watch mode), revisit by adding content-hash-based skip lists or gzip compression.
259288

260289
## NSSH1 Handshake Protocol
261290

‎build/test.toml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,3 +29,8 @@ run = "bash e2e/bash/test_port_forward.sh"
2929
description = "Run custom image build and sandbox e2e test"
3030
depends = ["cluster:deploy"]
3131
run = "bash e2e/bash/test_sandbox_custom_image.sh"
32+
33+
["test:e2e:sync"]
34+
description = "Run sandbox file sync e2e test"
35+
depends = ["cluster:deploy"]
36+
run = "bash e2e/bash/test_sandbox_sync.sh"

‎crates/navigator-cli/Cargo.toml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,9 @@ miette = { workspace = true }
4848
thiserror = { workspace = true }
4949
anyhow = { workspace = true }
5050

51+
# File archiving (tar-over-SSH sync)
52+
tar = "0.4"
53+
5154
# Streams
5255
futures = { workspace = true }
5356

‎crates/navigator-cli/src/main.rs‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -458,6 +458,25 @@ enum SandboxCommands {
458458
command: ForwardCommands,
459459
},
460460

461+
/// Sync files to or from a sandbox.
462+
Sync {
463+
/// Sandbox name.
464+
name: String,
465+
466+
/// Push local files up to the sandbox.
467+
#[arg(long, conflicts_with = "down", value_name = "LOCAL_PATH")]
468+
up: Option<String>,
469+
470+
/// Pull sandbox files down to the local machine.
471+
#[arg(long, conflicts_with = "up", value_name = "SANDBOX_PATH")]
472+
down: Option<String>,
473+
474+
/// Destination path (sandbox path when pushing, local path when pulling).
475+
/// Defaults to /sandbox for --up or . for --down.
476+
#[arg(value_name = "DEST")]
477+
dest: Option<String>,
478+
},
479+
461480
/// Manage sandbox images.
462481
Image {
463482
#[command(subcommand)]
@@ -821,6 +840,22 @@ async fn main() -> Result<()> {
821840
SandboxCommands::Create { .. } | SandboxCommands::Image { .. } => {
822841
unreachable!()
823842
}
843+
SandboxCommands::Sync {
844+
name,
845+
up,
846+
down,
847+
dest,
848+
} => {
849+
run::sandbox_sync_command(
850+
endpoint,
851+
&name,
852+
up.as_deref(),
853+
down.as_deref(),
854+
dest.as_deref(),
855+
&tls,
856+
)
857+
.await?;
858+
}
824859
SandboxCommands::Get { name } => {
825860
run::sandbox_get(endpoint, &name, &tls).await?;
826861
}

‎crates/navigator-cli/src/run.rs‎

Lines changed: 49 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,8 @@ use tonic::{Code, transport::Channel};
4242
// Re-export SSH functions for backward compatibility
4343
pub use crate::ssh::{list_forwards, stop_forward, stop_forwards_for_sandbox};
4444
pub use crate::ssh::{
45-
sandbox_connect, sandbox_exec, sandbox_forward, sandbox_rsync, sandbox_ssh_proxy,
45+
sandbox_connect, sandbox_exec, sandbox_forward, sandbox_ssh_proxy, sandbox_sync_down,
46+
sandbox_sync_up, sandbox_sync_up_files,
4647
};
4748

4849
/// Convert a sandbox phase integer to a human-readable string.
@@ -1168,11 +1169,12 @@ pub async fn sandbox_create(
11681169
let repo_root = git_repo_root()?;
11691170
let files = git_sync_files(&repo_root)?;
11701171
if !files.is_empty() {
1171-
sandbox_rsync(
1172+
sandbox_sync_up_files(
11721173
&effective_server,
11731174
&sandbox_name,
11741175
&repo_root,
11751176
&files,
1177+
"/sandbox",
11761178
&effective_tls,
11771179
)
11781180
.await?;
@@ -1447,6 +1449,51 @@ fn load_sandbox_policy(cli_path: Option<&str>) -> Result<SandboxPolicy> {
14471449
})
14481450
}
14491451

1452+
/// Sync files to or from a sandbox.
1453+
///
1454+
/// Dispatches to `sandbox_sync_up` or `sandbox_sync_down` based on the
1455+
/// `--up` / `--down` flags.
1456+
pub async fn sandbox_sync_command(
1457+
server: &str,
1458+
name: &str,
1459+
up: Option<&str>,
1460+
down: Option<&str>,
1461+
dest: Option<&str>,
1462+
tls: &TlsOptions,
1463+
) -> Result<()> {
1464+
match (up, down) {
1465+
(Some(local_path), None) => {
1466+
let sandbox_dest = dest.unwrap_or("/sandbox");
1467+
let local = Path::new(local_path);
1468+
if !local.exists() {
1469+
return Err(miette::miette!(
1470+
"local path does not exist: {}",
1471+
local.display()
1472+
));
1473+
}
1474+
eprintln!("Syncing {} -> sandbox:{}", local.display(), sandbox_dest);
1475+
sandbox_sync_up(server, name, local, sandbox_dest, tls).await?;
1476+
eprintln!("{} Sync complete", "✓".green().bold());
1477+
}
1478+
(None, Some(sandbox_path)) => {
1479+
let local_dest = Path::new(dest.unwrap_or("."));
1480+
eprintln!(
1481+
"Syncing sandbox:{} -> {}",
1482+
sandbox_path,
1483+
local_dest.display()
1484+
);
1485+
sandbox_sync_down(server, name, sandbox_path, local_dest, tls).await?;
1486+
eprintln!("{} Sync complete", "✓".green().bold());
1487+
}
1488+
_ => {
1489+
return Err(miette::miette!(
1490+
"specify either --up <local-path> or --down <sandbox-path>"
1491+
));
1492+
}
1493+
}
1494+
Ok(())
1495+
}
1496+
14501497
/// Fetch a sandbox by name.
14511498
pub async fn sandbox_get(server: &str, name: &str, tls: &TlsOptions) -> Result<()> {
14521499
let mut client = grpc_client(server, tls).await?;

0 commit comments

Comments
 (0)