This repository is the comparison layer for independent Qwen3.8-27B GGUF machine studies. It currently compares two completed Windows/CUDA measurements and reserves an explicitly empty Apple Silicon/macOS row.
Current evidence status: RTX 4090 measured; RTX 4070 Ti measured; Apple Silicon not measured. No Mac result is estimated or inferred.
Under the shared 4K baseline operating point (llama.cpp b10448, 84 prompt tokens, 256 completion tokens, one slot, q8_0 K/V, MTP off), the RTX 4090 measured:
| Quantization | RTX 4090 | RTX 4070 Ti | RTX 4090 uplift |
|---|---|---|---|
| UD-IQ2_XXS | 73.885 tok/s | 43.643 tok/s | 69.294% |
| UD-Q2_K_XL | 65.898 tok/s | 38.030 tok/s | 73.279% |
The IQ2 context ladder also used matching 3,231-, 6,423-, and 12,831-token prompts:
At the 16K configuration, the RTX 4090 produced 65.434 tok/s versus 39.201 tok/s on the RTX 4070 Ti, a 66.919% uplift.
- The 4K IQ2 and Q2 rows are the cleanest cross-machine performance comparison because their runtime, model identity, prompt, sampling controls, cache types, slot count, and full 66/66 CUDA layer placement match.
- The context chart is a matched IQ2 synthetic workload comparison, not a long-context reasoning benchmark.
- Objective-suite scores are retained in the normalized data for auditability, but they are not treated as GPU performance. IQ2 scored 8/24 on the RTX 4090 replication and 9/24 on the RTX 4070 Ti study; this one-task difference is not a hardware-quality claim.
- IQ4_XS is not charted across the GPUs: the RTX 4090 used 66/66 CUDA layers, while the RTX 4070 Ti used a hybrid 45/66 operating point. Comparing those values as though only the GPU changed would be misleading.
- MTP speedups are workload-specific and remain opt-in evidence. Both studies kept MTP off by default because at least one paired prose output changed.
- The Mac track stays
not_measuredwithnullresults until Phase 0 and later approved measurement phases run on the actual Apple Silicon machine.
Only Python's standard library is required:
python scripts/generate_comparison.py
python scripts/verify_comparison.pyThe generator reads the normalized machine records in data/machines, writes data/comparison.json, and regenerates the SVG figures. CI rejects stale generated files and invented Mac measurements.
See METHODOLOGY.md for inclusion rules and SOURCES.md for pinned upstream evidence.
- RTX 4090 / Windows / CUDA
- RTX 4070 Ti / Windows / CUDA
- Apple Silicon / macOS / Metal — prepared, not measured
Licensed under Apache-2.0.