An open source, high-performance AI Gateway and LLM proxy built in Rust.
Unified OpenAI-compatible API for OpenAI, Anthropic, Gemini, DeepSeek, and any OpenAI-compatible provider.

🦀 Rust • 🔌 OpenAI Compatible • 🗄️ etcd

Why Teams Use AISIX as Their Enterprise AI Gateway

• 🦀 Rust + Tokio — Extreme performance with low resource footprint; ships as a single binary
• 🔌 OpenAI-Compatible — One API to call all LLMs; drop-in replacement with zero code changes
• ⚡ Dynamic Config — Hot-reload via etcd; update models and keys without restarting
• 🛡️ Enterprise-Grade Control — API key auth, rate limiting (RPM / TPM / concurrent), and per-model access control
• 📊 Observability — OpenTelemetry distributed tracing and Prometheus metrics out of the box
• 🎨 Admin UI — Built-in management dashboard for models, API keys, and a chat playground

---

Features

🌐 Multi-Provider Support

Provider	Chat Completions	Streaming	Embeddings
🟢 OpenAI	✅	✅	✅
🟠 Anthropic	✅	✅	—
🔵 Gemini	✅	✅	✅
🐋 DeepSeek	✅	✅	—
🔌 OpenAI-Compatible	✅	✅	✅

🚦 Traffic Management

• Rate limiting — RPM, RPD, TPM, TPD, and concurrent request limits
• Per-model and per-key access control
• Request validation with JSON Schema

🛡️ Security & Auth

• API key authentication on all proxy requests
• Per-key model allowlist
• Admin API key protection

📊 Observability

• OpenTelemetry distributed tracing (Jaeger / Zipkin)
• Prometheus metrics export
• Structured logging via logforth

🎨 Admin & Management

• RESTful Admin API with OpenAPI spec + Scalar UI
• React-based Admin Dashboard
• Model CRUD / API Key CRUD / Chat Playground
• etcd-backed dynamic configuration (no restarts needed)

---

Architecture

---

Quick Start

curl -fsSL https://run.api7.ai/aisix/quickstart | sh

This script downloads configuration files, generates a random Admin Key, and starts AISIX with etcd via Docker Compose. Once running:

• Proxy API: http://127.0.0.1:3000
• Admin API: http://127.0.0.1:3001/aisix/admin
• Admin UI: http://127.0.0.1:3001/ui

For the full setup guide including model configuration and making your first request, see the Quick Start documentation.

---

Development

See CONTRIBUTING.md for the full contribution guide including branching, code style, and PR expectations.

Prerequisites

• Rust (latest stable)
• Node.js LTS + pnpm (corepack enable pnpm)
• Docker & Docker Compose (for etcd and the test environment)
• protobuf-compiler — sudo apt install protobuf-compiler (Debian/Ubuntu) or brew install protobuf (macOS)

Build & Run

1. Build UI

   cd ui
   pnpm install --frozen-lockfile
   pnpm build
   
   ## Or if you don't want to, then create a stub folder.
   ## Run this command in the root directory of the project.
   mkdir -p ui/dist

2. Build gateway

   cargo run

Roadmap

• Guardrails: content filtering and safety checks
• Load Balancing / Fallback: across providers
• Cost tracking & usage analytics
• Semantic caching: cache responses based on query intent, not exact match
• More providers: Azure, Bedrock, Ollama...
• Kubernetes Helm chart
• New protocol support
  • OpenAI Responses API
  • Anthropic Messages API
  • Google Gemini GenerateContent API
• Multimodal APIs: Image, audio, video
• MCP proxy

Community

• Contribute via CONTRIBUTING.md — setup, code style, and PR guidelines
• Join our Discord server for real-time chat with the community and maintainers
• Use GitHub Discussions for questions, ideas, and architecture discussions
• Use GitHub Issues for bug reports, feature requests, and actionable tasks
• Follow repository activity for ongoing documentation and product updates

---

License

This project is licensed under the Apache License 2.0.