Building production-grade AI systems that survive latency, cost, and unreliable model outputs
2 merged PRs in fastapi/fastapi — 101k⭐ — both reviewed & merged by @tiangolo, the creator of FastAPI
🐛 #15937 — Fixed a real production bug
SSE and JSONL streaming endpoints silently returned
|
📝 #15930 — Closed a docs gapAdded missing API reference page for
|
Both PRs went through full maintainer review cycles — test restructuring, precedence edge cases, and OpenAPI schema correctness — not drive-by patches.
+ I design and ship AI systems that sit between research ideas and real products
+ My focus isn't demos — it's systems that don't fall apart in productionI work at the intersection of LLMs, system design, and production engineering. While others chase perfect outputs, I build systems that degrade gracefully, handle token economics, and fail predictably.
Current Focus:
- 🔨 Converting large codebases into structured content pipelines
- ⚡ Building agentic systems with explicit validation & retry logic
- 🎯 Optimizing async backends for long-running AI workloads
- 📊 Reducing token cost & latency in multi-step LLM workflows
|
🤖 LLM Pipelines |
⚡ Agentic Systems |
🔄 Async Backends |
class AIEngineer:
def __init__(self):
self.focus = [
"RAG systems that degrade gracefully under bad inputs",
"Reducing token cost & latency in multi-step LLM workflows",
"Clear ownership over clever abstractions",
"Shipping > hype"
]
def philosophy(self):
return """
Systems should fail predictably.
Production beats perfection.
Code quality = maintainability + readability + reliability.
"""
def approach(self):
return {
"reliability": "Systems that fail predictably, not mysteriously",
"ownership": "Clear ownership beats clever abstractions",
"execution": "Shipping working code > endless optimization",
"tradeoffs": "Explicit decisions documented in code & docs"
}🟣 Repo2Viral — Production LLM System
Turning large GitHub repositories into usable documentation & video content
🎯 The Challenge
❌ Partial context causes hallucinations in LLM outputs
❌ Token costs explode on large repositories (100k+ LOC)
❌ Long-running jobs can't block HTTP requests
❌ Naive text chunking breaks code semantics
✅ The Solution
✓ Code-aware chunking & retrieval (AST-based, not naive splits)
✓ Async FastAPI workers for long-running jobs
✓ Rule-based validation before surfacing LLM output
✓ Structured prompt templates with explicit failure modes
✓ Token budget management per processing step
Architecture Decisions:
- FastAPI + Background Tasks: Decoupled long-running LLM calls from HTTP responses
- Supabase: Managed PostgreSQL for repo metadata & job status
- OpenAI API: GPT-4 for code understanding, GPT-3.5-turbo for content generation
- Next.js Frontend: Server-side rendering for SEO + client-side interactivity
Tech Stack: FastAPI Next.js OpenAI API Supabase Docker Redis
Key Metrics:
- ⚡ Handles repos up to 150k LOC
- 💰 ~60% token cost reduction via smart chunking
- 🎯 <3s API response time (job queuing, not blocking)
⚡ Agentic AI Image Studio — Latency-First Design
Multi-agent system for automated prompt refinement & output control
💡 Explicit Tradeoff: Speed vs Quality
Decision: Used Latent Consistency Models (LCM) instead of standard diffusion models
Impact:
- ✅ ~10× faster inference (2-4 steps vs 20-50 steps)
- ✅ Better throughput for production use cases
⚠️ Slight quality degradation vs SDXL/SD 2.1
Why this matters: Real products prioritize speed and reliability over perfect outputs. Users prefer fast, good-enough results to slow, perfect ones.
Implementation:
# Fast inference pipeline
pipe = DiffusionPipeline.from_pretrained(
"SimianLuo/LCM_Dreamshaper_v7",
scheduler=LCMScheduler()
)
# 4 steps instead of 50
images = pipe(prompt, num_inference_steps=4)🤖 Agent Architecture
Multi-Agent Orchestration:
- Prompt Refinement Agent: Enhances user input using GPT-3.5
- Image Generation Agent: LCM-based diffusion pipeline
- Quality Control Agent: Rule-based + CLIP scoring validation
- Retry Logic: Automatic regeneration on quality threshold failure
Tech Stack: PyTorch Diffusers LangChain CLIP FastAPI
🌿 Plant Disease Detection API — ML, Shipped Properly
Production-ready computer vision service with REST API
🎯 Production-First Design
✓ Fine-tuned ResNet50 (98% validation accuracy on PlantVillage dataset)
✓ Dockerized REST API with health checks & monitoring
✓ Designed for deployment, not just notebooks
✓ Input validation, error handling, and logging baked in
✓ <100ms inference latency on CPU
Why ResNet50?
- Proven architecture with ImageNet pretraining
- Excellent accuracy/speed tradeoff for deployment
- Smaller than ResNet101 → faster inference
- Well-supported in production frameworks (ONNX, TorchScript)
📊 Model Performance
| Metric | Value |
|---|---|
| Validation Accuracy | 98.2% |
| Inference Time (CPU) | ~95ms |
| Model Size | 102 MB |
| Classes | 38 diseases |
Tech Stack: PyTorch FastAPI Docker scikit-learn Pillow
API Endpoints:
POST /predict- Single image classificationPOST /batch- Batch predictionGET /health- Health checkGET /metrics- Model performance metrics
|
Capabilities:
Capabilities:
Capabilities: |
Capabilities:
Capabilities:
Capabilities:
|
90+ LeetCode Problems Solved 🎯
Focus Areas:
- 🌳 Data Structures: Arrays, Trees, Graphs, Hash Tables
- 🧮 Algorithms: Dynamic Programming, Greedy, DFS/BFS
- 🎯 System Design: Scalability, Caching, Load Balancing
- ⚡ Optimization: Time/Space complexity analysis
|
Data Science Intern | Remote Key Achievements: Tech Stack: |
Cloud Computing Intern | Remote Key Achievements: Tech Stack: |
|
Bachelor of Engineering in Computer Science Relevant Coursework:
|
|
| Principle | What It Means |
|---|---|
| 🎯 Reliability First | Systems should fail predictably, not mysteriously |
| 🏗️ Ownership > Cleverness | Clear, maintainable code beats smart abstractions |
| 🚀 Shipping > Perfection | Working code in production > perfect code in development |
| 📊 Data-Driven Decisions | Measure everything, optimize what matters |
| 🔧 Pragmatic Engineering | Choose the right tool for the job, not the coolest one |
- 🚀 Building Production-Ready RAG Systems: Lessons Learned
- ⚡ Optimizing LLM Token Costs in Multi-Agent Workflows
- 🎯 Why Your AI Demo Will Fail in Production (And How to Fix It)
- 🔧 FastAPI + Background Workers: Handling Long-Running AI Tasks
Building something interesting? Let's talk about systems that work in production.
I'm always interested in discussing:
- 🤖 Production LLM systems & RAG pipelines
- ⚡ Optimization strategies for AI workloads
- 🔧 System design & architecture decisions
- 🚀 Collaborative projects & open-source contributions
🎯 Systems Engineer • Not a Demo Builder
Focused on what survives production, not what looks good in slides.
⭐️ From SAURABHSALVE | Built with 💜 and lots of ☕