Put info about Rust toolchain into recipe and use it in cargo chef cook #231
Description
Problem
- Currently
cargo chef cookuses default Rust toolchain. - The project can define a Rust toolchain it wants to be built with using
rust-toolchain.toml. - This may lead to dependencies being built using different toolchain than the actual project. As a result,
cargo buildwill compile the dependencies again using the toolchain defined inrust-toolchain.toml.
The solution would be to include info about the toolchain into the recipe and use that info in cargo chef cook to make sure it compiles dependencies using the correct toolchain.
Problem reproducer:
cargo new --lib dummy- cd dummy
- Create
rust-toolchain.tomlwith following content:
[toolchain]
profile = "default"
channel = "1.70.0"
- Add
tokioas a dependency inCargo.tomlto be able to observe it being compiled twice:
[package]
name = "dummy"
version = "0.1.0"
edition = "2021"
[dependencies]
tokio = { version = "1", features = ["full"] }
- Create a
Dockerfile:
FROM rust:slim-bullseye AS chef
RUN cargo install cargo-chef
FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json
FROM chef AS builder
COPY --from=planner /recipe.json recipe.json
RUN cargo chef cook --recipe-path recipe.json
COPY . .
RUN cargo build
- Run
docker build --progress=plain --target builder --no-cache . 2>&1 | tee build-wrong.log - Observe dependencies being compiled both by
cargo chef cookandcargo build:
...
#10 [builder 2/4] RUN cargo chef cook --recipe-path recipe.json
#10 0.334 Updating crates.io index
#10 0.699 Downloading crates ...
#10 1.053 Downloaded parking_lot_core v0.9.8
#10 1.058 Downloaded bytes v1.4.0
#10 1.064 Downloaded signal-hook-registry v1.4.1
#10 1.065 Downloaded num_cpus v1.16.0
#10 1.072 Downloaded lock_api v0.4.10
#10 1.076 Downloaded tokio-macros v2.1.0
#10 1.080 Downloaded scopeguard v1.2.0
#10 1.083 Downloaded pin-project-lite v0.2.10
#10 1.095 Downloaded autocfg v1.1.0
#10 1.097 Downloaded socket2 v0.4.9
#10 1.100 Downloaded parking_lot v0.12.1
#10 1.103 Downloaded smallvec v1.11.0
#10 1.106 Downloaded mio v0.8.8
#10 1.128 Downloaded tokio v1.29.1
#10 1.189 Compiling libc v0.2.147
#10 1.189 Compiling autocfg v1.1.0
#10 1.189 Compiling proc-macro2 v1.0.66
#10 1.192 Compiling unicode-ident v1.0.11
#10 1.192 Compiling parking_lot_core v0.9.8
#10 1.192 Compiling scopeguard v1.2.0
#10 1.194 Compiling smallvec v1.11.0
#10 1.194 Compiling cfg-if v1.0.0
#10 1.230 Compiling pin-project-lite v0.2.10
#10 1.245 Compiling bytes v1.4.0
#10 1.430 Compiling lock_api v0.4.10
#10 1.430 Compiling tokio v1.29.1
#10 1.850 Compiling quote v1.0.32
#10 1.972 Compiling num_cpus v1.16.0
#10 1.972 Compiling mio v0.8.8
#10 1.974 Compiling socket2 v0.4.9
#10 1.974 Compiling signal-hook-registry v1.4.1
#10 1.980 Compiling syn v2.0.27
#10 2.121 Compiling parking_lot v0.12.1
#10 4.373 Compiling tokio-macros v2.1.0
#10 7.876 Compiling dummy v0.0.1 (/)
#10 9.263 Finished dev [unoptimized + debuginfo] target(s) in 8.96s
#10 DONE 9.4s
...
#12 [builder 4/4] RUN cargo build
#12 0.232 info: syncing channel updates for '1.70.0-x86_64-unknown-linux-gnu'
#12 0.472 info: latest update on 2023-06-01, rust version 1.70.0 (90c541806 2023-05-31)
#12 0.472 info: downloading component 'cargo'
#12 0.705 info: downloading component 'clippy'
#12 0.790 info: downloading component 'rust-docs'
#12 1.259 info: downloading component 'rust-std'
#12 2.162 info: downloading component 'rustc'
#12 4.139 info: downloading component 'rustfmt'
#12 4.230 info: installing component 'cargo'
#12 4.788 info: installing component 'clippy'
#12 5.059 info: installing component 'rust-docs'
#12 7.217 info: installing component 'rust-std'
#12 9.429 info: installing component 'rustc'
#12 14.03 info: installing component 'rustfmt'
#12 14.53 Compiling libc v0.2.147
#12 14.53 Compiling autocfg v1.1.0
#12 14.53 Compiling proc-macro2 v1.0.66
#12 14.53 Compiling unicode-ident v1.0.11
#12 14.53 Compiling parking_lot_core v0.9.8
#12 14.53 Compiling cfg-if v1.0.0
#12 14.53 Compiling smallvec v1.11.0
#12 14.53 Compiling scopeguard v1.2.0
#12 14.61 Compiling pin-project-lite v0.2.10
#12 14.64 Compiling bytes v1.4.0
#12 14.85 Compiling lock_api v0.4.10
#12 14.85 Compiling tokio v1.29.1
#12 15.34 Compiling quote v1.0.32
#12 15.49 Compiling mio v0.8.8
#12 15.49 Compiling socket2 v0.4.9
#12 15.49 Compiling signal-hook-registry v1.4.1
#12 15.49 Compiling num_cpus v1.16.0
#12 15.55 Compiling syn v2.0.27
#12 15.69 Compiling parking_lot v0.12.1
#12 18.01 Compiling tokio-macros v2.1.0
#12 21.67 Compiling dummy v0.1.0 (/)
#12 23.05 Finished dev [unoptimized + debuginfo] target(s) in 8.70s
#12 DONE 23.2s
...
Workaround
If we copy rust-toolchain.toml into chef image, it will make cargo chef cook to use the same Rust toolchain as the project being compiled:
FROM rust:slim-bullseye AS chef
RUN cargo install cargo-chef
COPY rust-toolchain.toml rust-toolchain.toml # <=== THIS IS ADDED
FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json
FROM chef AS builder
COPY --from=planner /recipe.json recipe.json
RUN cargo chef cook --recipe-path recipe.json
COPY . .
RUN cargo build
Now, when we build it with docker build --progress=plain --target builder --no-cache . 2>&1 | tee build-correct.log we will see only cargo chef cook compiling dependencies and cargo build not doing that.
Both logs attached
build-correct.log
build-wrong.log