Experimental CUDA port: Edge0-8B runs on NVIDIA T4 via MLX CUDA #20

I built an experimental CUDA fork of Edge0 and would welcome maintainer or community feedback on the approach.

Fork: https://github.com/ikaankeskin/Edge0-cuda

What currently works:

• Explicit EDGE0_BACKEND=cuda selection with no CPU fallback.
• Portable replacements for MLX CUDA operations unavailable in the pinned version:
  • 4-bit affine expert gather (GatherQMM)
  • dense quantized projections, embeddings, and tied output projections (QMM)
• Colab notebook for reproducible setup and diagnostics.
• Real Edge0-8B checkpoint generation on a Tesla T4, including 153 LoRA modules and 16 prerouter heads.

Validation:

• 98 tests passed, 1 skipped, 2 deselected.
• Two reset runs produced identical greedy token sequences.
• T4 warm run: 32 generated tokens in 82.2 seconds, including prefill.

Current limitations:

• Performance is not yet suitable for real-time use.
• Full Edge0-35B, Metal parity, long-context behaviour, and batched independent contexts are unverified.
• The merged fork needs a fresh CUDA run after incorporating newer upstream fixes.

I would especially value advice on:

1. Whether MLX CUDA is an acceptable direction for upstream.
2. The right long-term replacement for unavailable CUDA quantized kernels.
3. Any preferred benchmark or parity suite before proposing a PR.