Reproducible systems research and tooling for LLM serving, distributed training, GPU kernels, and AI networking.
modern-ai-infra is a modular, high-performance systems research toolkit and benchmark engine designed for deep-tech AI infrastructure optimization.
-
LLM Serving Engine & Simulator (
modern_ai_infra.llm_serving)- Event-driven Continuous Batching simulator (vLLM / TGI style scheduling).
- Paged KV-Cache block table allocator with internal & external memory fragmentation tracking and prefix cache sharing.
- Chunked Prefill & Speculative Decoding metrics (TTFT, ITL, P90/P99 tail latency distributions).
-
3D Parallelism & Pipeline Planner (
modern_ai_infra.distributed_training)- Analytical 3D Parallelism solver (Data Parallel / ZeRO-1, ZeRO-2, ZeRO-3, Tensor Parallelism, Pipeline Parallelism, Sequence Parallelism, Ring Attention).
- Per-GPU VRAM memory footprint breakdown (weights, gradients, Adam optimizer states, activations).
- Pipeline Schedule Simulator comparing 1F1B, Interleaved 1F1B, and Zero-Bubble pipeline schedules.
- Mixture-of-Experts (MoE) Expert Parallelism (EP) solver with capacity factor drops and Expert All-to-All throughput planner.
-
GPU Kernels & Hardware Roofline Analysis (
modern_ai_infra.gpu_kernels)- Hardware Roofline Analyzer (arithmetic intensity FLOP/Byte ceilings for H100 SXM, A100 80GB, B200 SXM, AMD MI300X, Apple Silicon Metal).
- FlashAttention-2 style fused tiled attention implementation in PyTorch with online softmax reduction.
- W8A16 / FP8 Quantized GEMM benchmark simulator evaluating memory compression ratio vs RMSE / SNR accuracy metrics.
-
AI Networking & Collective Operations (
modern_ai_infra.ai_networking)- Cluster Interconnect Topology solver (NVLink 4, NVSwitch, PCIe Gen5, Rail-Optimized InfiniBand NDR 400G Fat-Tree, RoCEv2).
- Analytical latency & throughput solver for collective primitives (Ring-AllReduce, Tree-AllReduce, ReduceScatter, AllGather, MoE All-to-All).
- CUDA stream Comm/Compute overlap efficiency estimator.
-
Interactive Web Dashboard (
dashboard/)- Modern glassmorphism web interface with real-time interactive roofline chart, 3D parallelism VRAM visualizer, serving timeline, and collective scaling curves.
# Clone repository
git clone https://github.com/Ritabanm/modern-ai-infra.git
cd modern-ai-infra
# Install in editable mode
pip install -e .The unified CLI tool ai-infra provides subcommands for each core pillar:
# 1. Run LLM Continuous Batching Serving Benchmark
ai-infra serve --num-requests 100 --qps 15.0 --num-blocks 512
# 2. Solve 3D Parallelism & Per-GPU VRAM Layout for Llama-3 8B / 70B
ai-infra train --model Llama-3-8B --gpus 64 --target-vram 80.0
# 3. Analyze Hardware Roofline & Run Fused FlashAttention Benchmark
ai-infra kernel --gpu nvidia_h100_sxm --seq-len 4096
# 4. Simulate Cluster Networking & Collective Communication Scaling
ai-infra net --nodes 4 --gpus-per-node 8
# 5. Run End-to-End Reproducibility Benchmark Suite & Export Report
ai-infra suite --output benchmark_report.jsonTo launch the interactive dashboard locally:
python3 -m http.server 8080 --directory dashboardOpen http://localhost:8080 in your browser to visually explore roofline models, 3D parallelism VRAM breakdowns, continuous batching timelines, and collective communication scaling graphs.
Run unit and integration tests with pytest:
python3 -m pytest tests/100% test pass rate across serving simulators, parallelism solvers, kernel rooflines, network topology models, and CLI runners.
MIT License. See LICENSE for details.