A self-hosted AI platform with Open WebUI, LiteLLM, and Ollama using Docker Compose. Supports both CPU-only (Docker Ollama) and GPU-accelerated (Native Mac Ollama) modes with easy switching.
Get your AI platform running in under 2 minutes:
# 1. Install dependencies
npm install
# 2. Configure environment
cp .env.example .env
# Edit .env if needed (defaults work fine)
# 3. Start the platform
npm run init
# 4. Access the web interface
# Open http://localhost:3000 in your browser
# Login with: [email protected] / admin_password123Expected Duration:
- Service startup: ~30 seconds
- Model download: 5-10 minutes (first time only)
- Total: ~10 minutes
- Docker Desktop - Download here
- Node.js 18+ - Download here
- For GPU acceleration (Option B): Ollama for Mac - Download here
βββββββββββββββ ββββββββββββββββ ββββββββββββββββ
β Browser βββββββΆβ Open WebUI βββββββΆβ LiteLLM β
β :3000 β β (Port 8080) β β Gateway β
β β β β β :4000 β
βββββββββββββββ ββββββββββββββββ ββββββββ¬ββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββ
β Option A: Docker Ollama :11434 β
β or β
β Option B: Native Ollama :11434 β
βββββββββββββββββββββββββββββββββββββ
- Open WebUI - ChatGPT-like interface with session history
- LiteLLM - API Gateway with key management and rate limiting
- Ollama - Inference engine (Dockerized or Native)
Edit .env to customize your platform:
# LiteLLM Configuration
LITELLM_MASTER_KEY=sk-admin-master-secret # Admin master key
LITELLM_PORT=4000 # Gateway port
# Open WebUI Configuration
WEBUI_PORT=3000 # Web UI port
WEBUI_ADMIN_EMAIL=[email protected] # Admin email
WEBUI_ADMIN_PASSWORD=admin_password123 # Admin password
ENABLE_SIGNUP=false # Allow new user signup
WEBUI_AUTH=true # Enable authentication
# Ollama Configuration
OLLAMA_PORT=11434 # Ollama API port
OLLAMA_MODE=docker # 'docker' or 'native'
# For native mode: Install Ollama.app and run: launchctl setenv OLLAMA_HOST "0.0.0.0"
# Model Configuration
DEFAULT_MODEL=llama3 # Default model to useOption A: Docker Ollama (CPU-only, Self-contained)
- Default mode
- No additional setup required
- CPU-only inference (slower)
- Isolated in Docker container
Option B: Native Ollama (GPU-accelerated, Faster)
- Requires Ollama for Mac
- 3-5x faster inference
- Uses Mac GPU acceleration
To switch from Docker to Native:
# 1. Install Ollama for Mac
brew install ollama # or download from ollama.com
# 2. Configure Ollama to listen on all interfaces
launchctl setenv OLLAMA_HOST "0.0.0.0"
# 3. Restart Ollama app completely (Quit + Reopen)
# 4. Pull model locally
ollama pull llama3
# 5. Update .env
echo "OLLAMA_MODE=native" >> .env
# 6. Restart platform
npm run teardown && npm run initTo switch from Native to Docker:
# 1. Update .env
echo "OLLAMA_MODE=docker" >> .env
# 2. Restart platform
npm run teardown && npm run init- Open http://localhost:3000
- Login with credentials from
.env - Select a model (default: llama3)
- Start chatting!
Generate API keys for programmatic access:
npm run gen-key
# Or with a custom username
npm run gen-key john_doeExample output:
π Generating API Key...
β
API Key Generated!
βββββββββββββββββββββββββββββββββββββββββββββββ
ποΈ API Key: sk-abc123...
π€ User: john_doe
π€ Model: llama3
βββββββββββββββββββββββββββββββββββββββββββββββ
π‘ Endpoint: http://localhost:4000/v1
βββββββββββββββββββββββββββββββββββββββββββββββ
Using curl:
curl http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama3",
"messages": [
{"role": "user", "content": "Hello!"}
]
}'Using Python:
import requests
response = requests.post(
'http://localhost:4000/v1/chat/completions',
headers={
'Authorization': 'Bearer YOUR_API_KEY',
'Content-Type': 'application/json'
},
json={
'model': 'llama3',
'messages': [
{'role': 'user', 'content': 'Hello!'}
]
}
)
print(response.json())Using Node.js:
const response = await fetch('http://localhost:4000/v1/chat/completions', {
method: 'POST',
headers: {
'Authorization': 'Bearer YOUR_API_KEY',
'Content-Type': 'application/json'
},
body: JSON.stringify({
model: 'llama3',
messages: [
{ role: 'user', content: 'Hello!' }
]
})
});
const data = await response.json();
console.log(data);Verify all services are operational:
npm run healthExpected output:
π₯ Health Check Report
Checking Docker...
Checking containers...
Checking endpoints...
βββββββββββββββββββββββββββββββββββββββββββββββ
β
Docker Daemon: healthy
β
ai-ollama: healthy
β
ai-gateway: healthy
β
ai-webui: healthy
β
Ollama API: healthy
β
LiteLLM API: healthy
β
Web UI: healthy
βββββββββββββββββββββββββββββββββββββββββββββββ
β
All systems operational!
Stop all services while preserving data:
npm run teardownStop and remove all data (including chat history and models):
npm run teardown -- --volumes- Model: Llama3 8B
- Tokens/second: 5-15 on M1 Pro/Max
- First response: 2-5 seconds
- Memory usage: 6-8GB
- Model: Llama3 8B
- Tokens/second: 20-50 on M1 Pro/Max
- First response: 0.5-2 seconds
- Memory usage: 8-10GB (GPU + system)
- 3-5x faster than Option A
Solution: Start Docker Desktop and wait for it to fully initialize
Solution: Change ports in .env:
WEBUI_PORT=3001
LITELLM_PORT=4001
OLLAMA_PORT=11435Solution:
- Check internet connection
- Verify disk space (need ~10GB free)
- Retry:
docker exec ai-ollama ollama pull llama3
Solution:
- Verify
OLLAMA_HOST=0.0.0.0is set - Restart Ollama app completely
- Test:
curl http://localhost:11434/api/version - Check from container:
docker exec ai-gateway curl http://host.docker.internal:11434
Solution:
# Check logs
docker compose logs
# Restart individual service
docker restart ai-gateway
# Full restart
npm run teardown && npm run init- Default credentials in
.env.example - Master key for admin access
- No HTTPS (localhost only)
- Generate strong random keys
- Enable HTTPS with reverse proxy (Caddy/Nginx)
- Use Docker secrets or external secret manager
- Enable rate limiting per user
- Regular security updates:
docker compose pull - Network isolation (Docker networks)
- Monitor usage via LiteLLM admin UI
minimal-ai-paas/
βββ Configuration
β βββ .env # Runtime config (git-ignored)
β βββ .env.example # Template
β βββ docker-compose.yml # Service orchestration
β
βββ LiteLLM Gateway
β βββ litellm/config.yaml # Model routing
β
βββ TypeScript Scripts (zx)
β βββ scripts/utils.ts # Shared utilities
β βββ scripts/init.ts # Setup & start
β βββ scripts/generate-key.ts # API key generation
β βββ scripts/health-check.ts # Service verification
β βββ scripts/teardown.ts # Clean shutdown
β
βββ Data (auto-created, git-ignored)
β βββ ollama/ # Model storage (~5GB)
β βββ openwebui/ # Chat history & users
β
βββ Documentation
βββ README.md # This file
βββ plans/poc-spec.md # Original specification
- Test thoroughly - Run all verification steps
- Explore Web UI - Create conversations, test different prompts
- Try API integration - Build a simple Python/Node.js app
- Experiment with models - Add llama3.2, mistral, codellama
- Performance tune - Switch to Option B for GPU acceleration
- Multi-user setup - Create API keys for team members
- Production prep - Implement security recommendations
| Script | Description |
|---|---|
npm run init |
Initialize and start the platform |
npm run health |
Check health of all services |
npm run teardown |
Stop all services |
npm run gen-key |
Generate a new API key |
This is a POC project. Feel free to extend and adapt it for your needs!
MIT License - feel free to use this project for any purpose.
- Open WebUI - Chat interface
- LiteLLM - API Gateway
- Ollama - Inference engine
- zx - Shell scripting in JavaScript