I'm an AI Engineer at Charlemagne Labs and a recent MS in Computer Engineering graduate from NYU, passionate about building efficient and scalable AI systems. My work focuses on LLM inference optimization (vLLM, quantization), GPU kernel programming (CUDA C++), RAG architectures, and multimodal models. An active open-source contributor—most recently to the 🤗 transformers library—I am currently seeking full-time Machine Learning Engineering roles to build high-performance AI infrastructure.
Languages: Python (vLLM, LlamaIndex, PyTorch, DeepSpeed, NumPy, Scikit-Learn, PySpark, TensorFlow), CUDA C++, SQL Domains: LLMs, RAG, Inference Optimization (Paged Attention, Chunked Prefix), Vision-Language Models, Quantization (bfloat16, AWQ, QAT), Distributed Training (FSDP, DDP), Phishing Detection & Security AI Tools: Docker, Kubernetes, Slurm, Hugging Face Transformers, LangChain, Ollama, GCP (Vertex AI, BigQuery), AWS, Apache (Spark, Airflow, Beam), Weights & Biases, CI/CD Pipelines
-
Inference & Model Optimization
- vLLM & Serving: Chunked Prefix Caching, Paged Attention, Flash Attention, Disaggregated Serving, Tensor Parallelism
- Quantization: Activation-aware Weight Quantization, SmoothQuant, Quantization-Aware Training (QAT), Dynamic Quantization, Native bfloat16 Optimization
- Techniques: Key-Value (KV) Caching, JIT Compilation, CUDA Kernels
-
Parameter-Efficient Fine-Tuning (PEFT) & Pruning
- PEFT Methods: LoRA, QLoRA, Soft Prompting (PEFT library)
- Pruning Strategies: NetAdapt-style structured pruning, Magnitude pruning, Filter/Channel pruning
- Optimization: Latency-aware model compression (e.g., 64% size reduction on DistilBERT) using Hugging Face Optimum
-
Distributed Training & MLOps
- Parallelism: Fully Sharded Data Parallel (FSDP), PyTorch DDP, Tensor Parallelism
- Memory Efficiency: DeepSpeed (ZeRO-Offload), Gradient Checkpointing, Mixed Precision Training
- Infrastructure: Multi-node cluster scaling (Slurm/Kubernetes), Docker containerization
-
Multimodal & RAG Architectures
- RAG Systems: LlamaIndex-based pipelines, Vector Search, Knowledge Retrieval, Faithfulness Evaluation
- Vision-Language: CLIP-style ViT+BERT alignment, Cross-modal Knowledge Distillation
- Architectures: ResNet + DistilBERT student-teacher networks, Transformer-based encoders
AI Engineer — Charlemagne Labs (Jan 2026 – Present) Building real-time phishing URL classification systems using small language models:
Fine-tuning a Gemma 3-based 270M-parameter SLM, improving precision/recall from 49%/90% to 96%/94% across 30+ model iterations Architected a teacher–student evaluation framework using a 9B-parameter LLM to identify and fix SLM weaknesses Engineered a multi-stage synthetic data pipeline producing 36K balanced training samples for robustness against unseen threats Deployed continuous evaluation on AWS against OpenPhish and Tranco ground-truth datasets
Production vLLM deployment of Mixtral 8x7B Instruct (bfloat16) across 4× A100 SXM4 80GB GPUs with full NVLink mesh: Tensor parallelism (TP=4) over NV12 NVLink — 600 GB/s bidirectional, confirmed via nvidia-smi topo Benchmarked 728 tok/s at concurrency 16 with 95–96% SM utilization uniformly across all GPUs Expert load analysis via MoE router hooks — 1.28× imbalance ratio, experts 0–1 absorbing 30.5% of tokens Automated provisioning pipeline: Vast.ai API → SSH → Ray cluster init → vLLM launch
A collection of GPU-accelerated kernels written in CUDA C++ for core ML and numerical operations:
Matrix Multiplication — tiled shared-memory matmul and sparse matmul (CSR format) Parallel Reductions — 4 progressive optimization stages (naive → warp-level) Prefix Sum — Blelloch-style inclusive/exclusive scan Dot Product & Vector Add — fundamental CUDA parallelism patterns CUDA Streams — overlapping computation with memory transfers Categorical Cross-Entropy — custom loss kernel for training pipelines
A local agentic RAG system for multi-step Q&A over SEC filings and structured financials for Apple, Microsoft, and Alphabet (FY2023–2025): ReAct agent with parallel tool calls — dynamically routes between SQL and PDF retrieval per query Hybrid dense + sparse retrieval (FAISS + BGE embeddings, BM25) fused with RRF, reranked via CrossEncoder SQLite backend covering income statements, balance sheets, segment & geographic revenue across 3 companies Multi-tier evaluation pipeline — fuzzy numeric scoring, entity matching, and LLM-based judging
-
#38487: Enable
device_map="auto"support for Dinov2
Enabled automatic device placement for Dinov2 by defining_no_split_modules, unlocking inference across CPU/GPU seamlessly. -
#38461: Add
GLPNImageProcessorFast
Implemented a fast image processor variant for the GLPN model using TorchVision. Achieved functional parity with the original (max abs diff < 1e-7) and added complete tests. -
#38509: SparseVLM – Visual Token Sparsification for Efficient VLM Inference
Proposed support for SparseVLM: a training-free, plug-and-play method to prune redundant image tokens in VLMs like BLIP and Flamingo.
It uses attention-guided token selection and recycling for up to 60% FLOPs reduction with minimal accuracy loss. Currently preparing an implementation compatible with 🤗transformers.
- 📧 Email: [email protected]
- 🌐 Portfolio: github.com/aryanchauhan31
Let’s build something impactful together.