This crate contains runnable examples for the user-facing API and kernel DSL.
Run these commands from the cutile-rs/ workspace, not the monorepo root.
Reusable kernel implementations live in cutile-kernels;
the QKV and MoE examples import those implementations and supply host-side
validation and timing code.
- Start with
cargo run -p cutile-examples --example hello_worldto verify the toolchain. - Run
cargo run -p cutile-examples --example saxpyfor a small kernel launch example. - Run
cargo run -p cutile-examples --example nvfp4for a CUDA 13.3 NVFP4 linear-tile example with output checking on native NVFP4 targets. - Run
cargo run --release -p cutile-examples --example nvfp4_moefor an NVFP4 mixture of experts benchmark with device routing, activation quantization, fused FC1/ReLU-squared/requantization, FC2, and weighted finalization. It checks outputs and reports CUDA graph latency per batch size. - Run
cargo run --release -p cutile-examples --example fused_qkvfor fused QKV projection, Q/K RMSNorm, FP8 conversion and paged KV stores. Add-- --benchfor the opt-in graph timing sweep; see the QKV guide for layouts, validation scope and tuning options. - Run
cargo run -p cutile-examples --example pdlfor a CUDA 13.4 programmatic dependent launch example: a producer/consumer kernel pair with output checking and a stream-order vs PDL timing comparison (sm_90 or newer). - Run
cargo run -p cutile-examples --example async_gemmfor a larger async example. - Run
cargo run --release -p cutile-examples --example autotune --features experimental-tunefor RMSNorm tuning with CPU compilation overlapped with GPU evaluation. It checks correctness and prepared-binary reuse;-- --workers 4increases preparation concurrency and-- --serialdisables overlap for comparison. See the autotuning guide for deadlines and finalist-confirmation requirements.
Every example checks the selected tileiras and device before allocating
buffers or launching kernels. An unmet requirement prints SKIP <example>:
with the required and selected capabilities, then exits successfully.
Tool discovery, compilation, execution and validation errors still fail.
| Examples | Minimum requirements |
|---|---|
| All except those listed below | Tile IR 13.2 and a supported SM80+ target |
nvfp4, nvfp4_moe, mxfp8 |
Tile IR 13.3 and SM100+ (including SM120/121) |
fused_qkv |
FP8-capable SM90+; SM90 requires Tile IR 13.3, while SM100/120 support 13.2 |
pdl |
Tile IR 13.4 and SM90+; the driver must support the extended launch API |
The compiler's target rules also apply: SM90 requires Tile IR 13.3 and SM107
requires 13.4. These checks live in src/requirements.rs; they are not Cargo
features and do not change which APIs are compiled.
nvfp4_moe measures the full pipeline's CUDA graph replay latency for batch
sizes 1, 2, 4, 8, 16, 32, 64, 128, and 256. Each batch uses 20 samples of
1,000 replays, with 50 warmups per sample. Compilation, allocation, transfers,
and output validation are outside the timed region. The original default
fixture remains H=I=1024, eight allocated experts, four active experts, top-k=4.
Direct launches and graph replay are checked against a deterministic reference.
The example imports shared kernels.
The opt-in B200 profiles provide SM100 schedules for evaluation with CUDA 13.5 tooling. Compiler support for individual schedules can vary; selecting this profile does not guarantee that every schedule is supported by every assembler build. The minimum execution requirements above remain unchanged for default profiles.
Select matching public toolkit tools and libraries explicitly, and remove inherited overrides from other toolkit installations:
export CUDA_TOOLKIT_PATH=/absolute/path/to/public/cuda-13.5
export CUDA_HOME="$CUDA_TOOLKIT_PATH"
export CUDA_PATH="$CUDA_TOOLKIT_PATH"
export CUDA_ROOT="$CUDA_TOOLKIT_PATH"
export CUDACXX="$CUDA_TOOLKIT_PATH/bin/nvcc"
export CUTILE_TILEIRAS_PATH="$CUDA_TOOLKIT_PATH/bin/tileiras"
export CUTILE_BYTECODE_VERSION=13.5
export PATH="$CUDA_TOOLKIT_PATH/bin:$PATH"
export LD_LIBRARY_PATH="$CUDA_TOOLKIT_PATH/nvvm/lib64:$CUDA_TOOLKIT_PATH/lib64:$CUDA_TOOLKIT_PATH/lib"
export LIBRARY_PATH="$CUDA_TOOLKIT_PATH/lib64:$CUDA_TOOLKIT_PATH/lib"
unset LD_PRELOAD
export LIBCLANG_PATH=/absolute/path/to/libclang/directory
nvcc --version
"$CUTILE_TILEIRAS_PATH" --version
"$CUTILE_TILEIRAS_PATH" --list-versions
sha256sum "$CUTILE_TILEIRAS_PATH"Preserve the repository's Rust toolchain pin.
Record actual backend-library loading during compilation (for example with
LD_DEBUG=libs), not just the version-command output.
From cutile-rs/:
cargo run --release --locked -p cutile-examples --example fused_qkv -- --bench --profile b200
cargo run --release --locked -p cutile-examples --example fused_qkv -- --bench --profile b200 --nontrivial
cargo run --release --locked -p cutile-examples --example nvfp4_moe -- --profile b200
cargo run --release --locked -p cutile-examples --example nvfp4_moe -- --profile b200 --shape large--profile b200 is explicit and requires SM100. Omitting it preserves the
original schedules; it is never selected based on a compiler filename.
MoE uses bounded persistent FC1/FC2 grids and per-batch tiles, occupancy,
latency and finalization settings. It still processes every device-computed
expert group; launch bounds do not assume four active experts.
--shape large selects H=6144, I=1536 and 192 allocated experts without a source
edit. Both shapes retain the same synthetic generator, top-k=4 and four active
experts. Label the shapes separately when reporting results. MoE prints every
selected schedule and all 20 sample_csv,batch,sample,latency_us records.
Timing still covers routing, quantization, FC1/ReLU-squared/requantization, FC2,
and weighted finalization.
For focused correctness/sanitizer runs, use --batch 16 --check-only, with
either shape or profile. This validates direct launches and graph replay
without timing. It does not suppress checks or change tolerances.
An assembler may reject individual B200 schedules, including TN=32. Record the
selected schedule and compiler diagnostics when comparing compilers; silently
substituting another profile changes the comparison. --profile default retains
the original launch settings.
Run correctness and sanitizer checks separately from timing:
cargo build --release --locked -p cutile-examples --example fused_qkv --example nvfp4_moe
CUTILE_JIT_SANITIZE=memcheck "$CUDA_TOOLKIT_PATH/bin/compute-sanitizer" \
--tool memcheck --error-exitcode 99 target/release/examples/fused_qkv
CUTILE_JIT_SANITIZE=memcheck "$CUDA_TOOLKIT_PATH/bin/compute-sanitizer" \
--tool memcheck --error-exitcode 99 target/release/examples/nvfp4_moe \
--shape large --batch 16 --check-onlyQKV keeps its original unpadded M=67 tail/scatter fixture. Its masked slot load also uses in-bounds addresses for inactive rows. Keep output checks enabled for both direct launches and graph replay.
Separately, on RTX 5090 (sm_120, driver 610.43.02), public tileiras 13.5.37
fails the original MoE direct-launch check at batch 16: normalized maximum error
0.5, maximum absolute error 1.578125. This reproduces with 13.4 and 13.5 bytecode.
Its cause remains unknown. This native numerical failure did not reproduce
in the B200 evaluations.
Retain all output checks.
To check compatibility, select the actual older assembler:
CUTILE_TILEIRAS_PATH=/usr/local/cuda-13.2/bin/tileiras scripts/run_examples.sh
CUTILE_TILEIRAS_PATH=/usr/local/cuda-13.3/bin/tileiras scripts/run_examples.shRun those commands from cutile-rs/. The runner counts passes,
skips and failures separately and shows error output for failures. It enables
reference-cpu,experimental-tune by default so all examples can be built.
On aarch64, the CPU-reference dependency may also need
RUSTFLAGS="-C target-cpu=native"; see the comments in Cargo.toml.
An example that only uses baseline features must keep running on 13.2/13.3. Do not raise its requirements to hide a compatibility regression.
The cutile-kernels example
provides correctness-checked host launches for GEMM memory variants, causal
attention specializations and routed FP16 MoE. Their implementations live in
cutile-kernels; no duplicate device kernels are maintained here.