Skip to content

About

Reproducible systems research and tooling for LLM serving, distributed training, GPU kernels, and AI networking.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

Repository files navigation

modern-ai-infra

Reproducible systems research and tooling for LLM serving, distributed training, GPU kernels, and AI networking.

PyTorch License Tests


Highlights & Pillars

modern-ai-infra is a modular, high-performance systems research toolkit and benchmark engine designed for deep-tech AI infrastructure optimization.

  1. LLM Serving Engine & Simulator (modern_ai_infra.llm_serving)

    • Event-driven Continuous Batching simulator (vLLM / TGI style scheduling).
    • Paged KV-Cache block table allocator with internal & external memory fragmentation tracking and prefix cache sharing.
    • Chunked Prefill & Speculative Decoding metrics (TTFT, ITL, P90/P99 tail latency distributions).
  2. 3D Parallelism & Pipeline Planner (modern_ai_infra.distributed_training)

    • Analytical 3D Parallelism solver (Data Parallel / ZeRO-1, ZeRO-2, ZeRO-3, Tensor Parallelism, Pipeline Parallelism, Sequence Parallelism, Ring Attention).
    • Per-GPU VRAM memory footprint breakdown (weights, gradients, Adam optimizer states, activations).
    • Pipeline Schedule Simulator comparing 1F1B, Interleaved 1F1B, and Zero-Bubble pipeline schedules.
    • Mixture-of-Experts (MoE) Expert Parallelism (EP) solver with capacity factor drops and Expert All-to-All throughput planner.
  3. GPU Kernels & Hardware Roofline Analysis (modern_ai_infra.gpu_kernels)

    • Hardware Roofline Analyzer (arithmetic intensity FLOP/Byte ceilings for H100 SXM, A100 80GB, B200 SXM, AMD MI300X, Apple Silicon Metal).
    • FlashAttention-2 style fused tiled attention implementation in PyTorch with online softmax reduction.
    • W8A16 / FP8 Quantized GEMM benchmark simulator evaluating memory compression ratio vs RMSE / SNR accuracy metrics.
  4. AI Networking & Collective Operations (modern_ai_infra.ai_networking)

    • Cluster Interconnect Topology solver (NVLink 4, NVSwitch, PCIe Gen5, Rail-Optimized InfiniBand NDR 400G Fat-Tree, RoCEv2).
    • Analytical latency & throughput solver for collective primitives (Ring-AllReduce, Tree-AllReduce, ReduceScatter, AllGather, MoE All-to-All).
    • CUDA stream Comm/Compute overlap efficiency estimator.
  5. Interactive Web Dashboard (dashboard/)

    • Modern glassmorphism web interface with real-time interactive roofline chart, 3D parallelism VRAM visualizer, serving timeline, and collective scaling curves.

Installation & Setup

# Clone repository
git clone https://github.com/Ritabanm/modern-ai-infra.git
cd modern-ai-infra

# Install in editable mode
pip install -e .

Command Line Interface (ai-infra)

The unified CLI tool ai-infra provides subcommands for each core pillar:

# 1. Run LLM Continuous Batching Serving Benchmark
ai-infra serve --num-requests 100 --qps 15.0 --num-blocks 512

# 2. Solve 3D Parallelism & Per-GPU VRAM Layout for Llama-3 8B / 70B
ai-infra train --model Llama-3-8B --gpus 64 --target-vram 80.0

# 3. Analyze Hardware Roofline & Run Fused FlashAttention Benchmark
ai-infra kernel --gpu nvidia_h100_sxm --seq-len 4096

# 4. Simulate Cluster Networking & Collective Communication Scaling
ai-infra net --nodes 4 --gpus-per-node 8

# 5. Run End-to-End Reproducibility Benchmark Suite & Export Report
ai-infra suite --output benchmark_report.json

Running Interactive Web Dashboard

To launch the interactive dashboard locally:

python3 -m http.server 8080 --directory dashboard

Open http://localhost:8080 in your browser to visually explore roofline models, 3D parallelism VRAM breakdowns, continuous batching timelines, and collective communication scaling graphs.


Automated Test Suite

Run unit and integration tests with pytest:

python3 -m pytest tests/

100% test pass rate across serving simulators, parallelism solvers, kernel rooflines, network topology models, and CLI runners.


License

MIT License. See LICENSE for details.

About

Reproducible systems research and tooling for LLM serving, distributed training, GPU kernels, and AI networking.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages