Skip to content

feat(builder, cargo-shuttle): build and run with docker locally#2044

Merged
jonaro00 merged 17 commits intomainfrom
builder
Sep 9, 2025
Merged

feat(builder, cargo-shuttle): build and run with docker locally#2044
jonaro00 merged 17 commits intomainfrom
builder

Conversation

@jonaro00
Copy link
Member

@jonaro00 jonaro00 commented May 15, 2025

Introduces (hidden commands + prints a warnings about being experimental):

  • shuttle build which builds the runtime natively with cargo build, just like shuttle run does.
    • Supports --release/-r and --bacon args from run
    • Supports --output-archive
  • shuttle build --docker which uses the same archiving logic as deploy, and the same dockerfile pipeline as the platform.
    • Great for debugging the platform's build system.
    • --tag/-t [tag] can be used to tag the image (in addition to the default image tag shuttle-build-[project_name]) so that it can later be used to docker push somewhere. Enables adding a --push arg in the future.
    • Does not support --bacon.
    • Always builds in release mode (like platform) (can be changed).
  • (experimental) shuttle run --docker does a local run with the docker build followed by a docker run.
    • Great for debugging the platform's runtime environment.
    • Does not support same things as build --docker, and potentially more. (alpha state, highly untested)

Adds aliases:

  • shuttle i for init
  • shuttle b for build (parity with cargo)
  • shuttle r for run (parity with cargo)
  • shuttle d for deploy (note: shuttle depl is current alias for deployment subcommands)

Also:

  • Fancier "Packing", "Uploading", "Building" printouts that match the ones from cargo
  • Let cargo build process print all its output to stdout/stderr, no tx/rx channels
  • Unify how args for local build and docker (remote) builds are gathered.

TODO (followups):

  • Increase parity in run --docker
  • CI for build & run with --docker

@jonaro00 jonaro00 changed the title wip: feat(builder): builder crate, shuttle build command wip: feat(builder): builder crate, build and run with docker locally Jun 3, 2025
@jonaro00 jonaro00 changed the title wip: feat(builder): builder crate, build and run with docker locally feat(builder): builder crate, build and run with docker locally Jun 3, 2025
@jonaro00 jonaro00 changed the title feat(builder): builder crate, build and run with docker locally feat(builder, cargo-shuttle): build and run with docker locally Jun 12, 2025
@jonaro00 jonaro00 force-pushed the builder branch 2 times, most recently from f950387 to 731e51f Compare June 18, 2025 18:21
@jonaro00 jonaro00 force-pushed the builder branch 6 times, most recently from 006e357 to 76ec5ca Compare July 2, 2025 15:39
@jonaro00 jonaro00 force-pushed the builder branch 2 times, most recently from cf77c16 to 52aa59e Compare July 4, 2025 21:59
@jonaro00 jonaro00 force-pushed the builder branch 4 times, most recently from 3c04294 to 24c6cd8 Compare July 28, 2025 10:56
@jonaro00 jonaro00 marked this pull request as ready for review September 5, 2025 16:35
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Summary

This PR introduces comprehensive Docker-based local build and run capabilities to the Shuttle CLI, allowing developers to replicate the platform's build environment locally for debugging purposes. The changes add two major new experimental commands: shuttle build (with optional --docker flag) and enhanced shuttle run --docker functionality.

The implementation centers around a new shuttle-builder crate that uses Askama templates to generate Dockerfiles dynamically based on project configuration. The builder creates multi-stage Docker builds with cargo-chef for dependency caching, matching the platform's production pipeline. Key architectural changes include refactoring build argument handling through a new BuildArgsShared struct that unifies configuration between local and Docker builds, and streamlining cargo output by removing async channel communication in favor of direct stdout/stderr forwarding.

The PR also enhances user experience with cargo-style command aliases (shuttle b, shuttle r, etc.) and improved status messages that match cargo's formatting conventions. The Docker build system supports flexible configuration including custom tags, binary selection, feature flags, and asset copying through Shuttle.toml integration. All Docker-related functionality is currently hidden behind experimental flags with appropriate warnings, indicating this is an early-stage feature being gradually introduced.

Important Files Changed

Changed Files
Filename Score Overview
common/src/models/deployment.rs 5/5 Added Debug trait to BuildArgsRust struct for better debugging support
cargo-shuttle/src/util/mod.rs 5/5 Added cargo_green_eprintln utility function for cargo-style output formatting
Cargo.toml 5/5 Added new shuttle-builder crate to workspace with askama templating dependency
builder/templates/cargo-chef.Dockerfile 4/5 New Dockerfile template for cargo-chef build environment with comprehensive build tools
cargo-shuttle/tests/integration/builder.rs 4/5 Refactored tests to use new cargo_build function instead of build_workspace
cargo-shuttle/tests/integration/run.rs 4/5 Simplified RunArgs initialization using Default trait for cleaner test code
builder/src/lib.rs 4/5 New Docker builder library with template rendering and hardcoded image names
cargo-shuttle/src/args.rs 4/5 Major refactoring adding Build command, Docker support, and command aliases
builder/Cargo.toml 4/5 New crate manifest with askama and shuttle-common dependencies
builder/templates/runtime-base.Dockerfile 4/5 Minimal Debian runtime base image template with essential packages
cargo-shuttle/src/lib.rs 4/5 Extensive changes adding build/run Docker commands and unified argument handling
builder/templates/rust.Dockerfile.jinja2 4/5 Comprehensive Jinja2 template for multi-stage Rust Docker builds
builder/tests/rust.Dockerfile 4/5 Test reference file for validating Dockerfile template rendering
cargo-shuttle/Cargo.toml 5/5 Added shuttle-builder dependency for Docker build functionality
cargo-shuttle/src/builder.rs 4/5 Refactored build process with modular functions and direct cargo output

Confidence score: 3/5

  • This PR introduces significant new functionality that is experimental and affects core build/run workflows with potential Docker integration issues
  • Score reflects the complexity of Docker build pipelines, template rendering, and the experimental nature of the features with limited testing mentioned
  • Pay close attention to builder/src/lib.rs for hardcoded image names and unwrap calls, cargo-shuttle/src/lib.rs for Docker command construction, and the Dockerfile templates for proper multi-stage build configuration

Sequence Diagram

sequenceDiagram
    participant User
    participant CLI as "Shuttle CLI"
    participant Builder as "Builder Module"
    participant Docker
    participant CargoMetadata as "Cargo Metadata"
    participant FileSystem as "File System"
    participant DockerfileTemplate as "Dockerfile Template"

    User->>CLI: "shuttle build --docker"
    CLI->>CLI: "Load local config"
    CLI->>Builder: "async_cargo_metadata()"
    Builder->>CargoMetadata: "Get project metadata"
    CargoMetadata-->>Builder: "Project metadata"
    Builder->>Builder: "gather_rust_build_args()"
    Builder-->>CLI: "Build arguments"
    
    CLI->>CLI: "Create temp directory"
    CLI->>CLI: "gather_build_files()"
    CLI->>FileSystem: "Copy build files to temp dir"
    FileSystem-->>CLI: "Files copied"
    
    CLI->>DockerfileTemplate: "render_rust_dockerfile()"
    DockerfileTemplate-->>CLI: "Generated Dockerfile"
    CLI->>FileSystem: "Write Dockerfile to temp dir"
    
    CLI->>Docker: "docker buildx build --tag shuttle-build-{project}"
    Docker->>Docker: "Build image with cargo-chef and runtime setup"
    Docker-->>CLI: "Build complete"
    
    alt Run with Docker
        User->>CLI: "shuttle run --docker"
        CLI->>CLI: "local_docker_build()"
        CLI->>Docker: "docker run --network host shuttle-build-{project}"
        Docker->>Docker: "Start container with runtime"
        Docker-->>CLI: "Container running"
        CLI->>User: "Service available on port"
    end

    alt Native Build
        User->>CLI: "shuttle build"
        CLI->>Builder: "cargo_build()"
        Builder->>Builder: "Execute cargo build command"
        Builder-->>CLI: "Built executable path"
        CLI->>User: "Build complete"
    end

    alt Native Run
        User->>CLI: "shuttle run"
        CLI->>Builder: "local_build()"
        Builder->>Builder: "cargo_build()"
        Builder-->>CLI: "Built service"
        CLI->>CLI: "Start provisioner server"
        CLI->>CLI: "Spawn runtime process"
        CLI->>User: "Service running locally"
    end
Loading

15 files reviewed, 3 comments

Edit Code Review Bot Settings | Greptile

@jonaro00 jonaro00 merged commit 764a3cd into main Sep 9, 2025
24 checks passed
@jonaro00 jonaro00 deleted the builder branch January 9, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants