KeyCompute

简体中文 | English | 繁體中文 | Español | العربية

Next-generation high-performance AI token compute service platform

Features • Architecture • Quick Start • Configuration • Project Structure • API • Development

Overview

KeyCompute is a high-performance, extensible, and out-of-the-box AI token compute service platform, providing enterprise-grade capabilities including unified LLM access, smart routing, metering and billing, compute node leasing, multi-level distribution, and observability.

Pure Rust Full Stack: Backend (Axum) + Frontend (Dioxus WASM) + CLI client, sharing types and logic, ultimate performance and security.

Note: This project is for personal learning only. You must use it in compliance with OpenAI Terms of Use and applicable laws and regulations. Do not use it for illegal purposes. In accordance with the Interim Measures for the Administration of Generative Artificial Intelligence Services, do not provide any unregistered generative AI services to the public in China.

Features

Compute Node Leasing

Compute nodes connect via pull-based polling without requiring a public IP. They run hosted models on local hardware and earn rewards based on contributions.

One-click connection: Run the standalone CLI binary to auto-register → heartbeat → poll tasks → local execution → submit results
NodeDispatch routing: Use /nt/v1/chat/completions with the raw model ID (for example gemma3:270m); the URL selects the node pool
Automatic failover: Failed nodes are excluded from scheduling, tasks are automatically requeued
Session persistence: Local sessions prevent duplicate registration; graceful shutdown ensures task integrity
Tip mechanism: Node owners can earn and withdraw tips

Unified Multi-model Gateway

Seamlessly switch between all major models with the standard OpenAI API — just one line of code:

Provider	Model Families	Implementation
🟢 OpenAI	GPT-4o / GPT-4 / GPT-3.5 etc.	✅
🟣 Anthropic	Claude 3.5 Sonnet / Opus / Haiku etc.	✅
🔵 Google	Gemini 1.5 / 2.0 Flash / Pro etc.	✅
🔴 DeepSeek	DeepSeek-V3 / R1 / Chat etc.	✅
🟤 Ollama	Local models (Llama / Qwen / GLM / MiniMax etc.)	✅
🟡 vLLM	Self-hosted models	✅

GLM (Zhipu) and MiniMax can be deployed locally via the Ollama adapter, not as standalone Provider implementations.

Smart Routing Engine

Two-layer routing architecture with multi-factor weighted scoring for optimal selection:

score = 0.30 × Cost Factor + 0.25 × Latency Factor + 0.25 × Success Rate + 0.20 × Health Status

Model-level routing → Account pool routing: Automatically distributes across providers and accounts
Fallback chain: Automatically switches to backup targets when primary target fails
Exponential backoff retry: Up to 3 retries, initial 100ms, max 10s
Request-level proxy: Supports provider-level / account-level / wildcard HTTP proxies

Billing & Payment System

Post-stream settlement: Precise calculation after request completion, no pre-deduction, no impact on results
Three-tier pricing: Tenant-specific pricing → Database default → Hardcoded fallback (LRU cache)
Precise usage: Priority to provider-precise usage, falls back to tiktoken estimation
Online top-up: Alipay/WeChat Pay + balance management
Usage analytics: Detailed token consumption breakdowns with visualization

Referral Distribution System

Referral commissions: Default 3% for first level + 2% for second level, auto-calculated
Invite links: Generate exclusive invite links with one click
Flexible configuration: Admins configure distribution ratios via API
Revenue analytics: View referral earnings and referral list in real time

Authentication & Permissions

Dual authentication: JWT (user sessions) + API Key (sk-..., API access)
Permission separation: API Key with admin role cannot access management interface
Complete user management: Registration → Email verification → Login → Password reset → Explicit platform/tenant role management
Explicit tenant membership: Registration creates a global identity without a tenant; an invitation adds a membership, and users can switch among verified memberships.
Group-based rate limiting: User-level / tenant-level / API Key-level throttling (in-memory / Redis dual backend)
Generation quota semantics: Generation RPM is charged once per execution on the authenticated tenant/user/API-key bucket; a selected account may tighten the tenant limit, while fallback attempts remain part of the same logical request. Account limits intentionally use that caller-scoped bucket (they are not an aggregate upstream-account bucket). Malformed requests, routing failures, and completed idempotency replays do not consume execution RPM.

Observability

Prometheus metrics: Request volume, latency, error rate, provider health
Distributed tracing: Provider Span / Request Span / Stream Span
Structured logging: JSON format, development/production tiered output
Host monitoring: CPU / Memory / Disk / Network real-time metrics
Health check: /health endpoint for one-click service status monitoring

Cross-platform Frontend

Web admin dashboard: Dioxus WASM SPA
Desktop: Dioxus Desktop native application
Mobile: Dioxus Mobile cross-platform support
Route-level permission control: verified platform capabilities and tenant memberships, with each management page scoped to the active identity

Architecture

[Client: Web / Desktop / Mobile (Dioxus)]
                ↕ HTTP/SSE
[API Layer: keycompute-server (Axum)]
       ├── Authentication (JWT + API Key)
       ├── Rate Limiting (In-memory/Redis)
       ├── Routing (Two-layer engine)
       └── Gateway (Single upstream execution layer)
                ↕
[Provider Adapter Layer]
  ├── OpenAI / Anthropic / Google
  ├── DeepSeek
  ├── Ollama (Local models)
  └── vLLM (Self-hosted)

[Compute Node Network]
  node-token (CLI) ↔ node-gateway ↔ Redis task queue ↔ Local inference

Quick Start

Requirements

Component	Version Requirement
Rust	≥ 1.92
Axum	≥ 0.8.0
Dioxus	0.7.9 (frontend development)
PostgreSQL	≥ 16
Redis	≥ 7 (optional, for distributed rate limiting/node queue)
Docker	Latest (container deployment)

Option 1: Docker Compose deployment (recommended)

# Clone the project
git clone https://github.com/keycompute/keycompute.git
cd keycompute

# Copy and edit environment variables
cp .env.example .env
# Before the first production startup, replace every operational credential.
# The application will refuse to start unless at least these values satisfy:
# - KC__AUTH__JWT_SECRET: non-default, non-blank, at least 32 bytes
# - KC__CRYPTO__SECRET_KEY: Base64 encoding of exactly 32 bytes
# - KC__NODE_GATEWAY__REGISTRATION_TOKEN_SECRET: non-default, at least 16 bytes
#   (required here because this Compose stack enables Redis/Node Gateway)
# - KC__DEFAULT_ADMIN_PASSWORD: non-default, non-blank, at least 12 characters
#   (required only while creating the first platform root identity)

# Start all services
docker compose up -d

# Check service status
docker compose ps

After deployment, visit http://localhost to get started (or use the port set by WEB_PORT).

Unless overridden, the bootstrap administrator email is [email protected]. Its password is the value you set in KC__DEFAULT_ADMIN_PASSWORD; production never accepts change-me-admin-password for a fresh database. After the first platform root identity has been created, remove this one-time password from the environment; later restarts do not require it.

Option 2: Local development

⚠️ Security Warning: The default values shown below (change-me-*) are for demonstration only. Never use these in production! Generate strong random passwords using:
openssl rand -base64 32
An ordinary cargo run builds the debug executable, so the startup method itself selects development mode. It reads only config.toml, may use public local credentials, and may listen on 0.0.0.0. .env.example, KC__*, and APP_BASE_URL cannot override that file. Never reuse it in production.

# Create the cargo-run configuration
cp config.example.toml config.toml

# Start PostgreSQL and both Redis services with host-local ports matching config.toml
docker compose --env-file .env.example -f docker-compose.yml -f docker-compose.dev.yml up -d postgres redis redis-cache

# Install the CLI version used by this workspace
cargo install dioxus-cli --version 0.7.9 --locked

# Start the backend
cargo run -p keycompute-server

# Start the frontend development server (in another terminal)
dx serve --package web --platform web --hot-reload true --addr 0.0.0.0

cargo run -p keycompute-server --release and the Docker image select production mode. The release executable reads environment variables and ignores config.toml; no configuration parameter or environment variable can change the runtime mode.

Project Structure

keycompute/
├── crates/                          # Backend core modules (Rust)
│   ├── keycompute-server/            # Axum HTTP service (integrates all modules)
│   ├── keycompute-types/             # Shared types and macros
│   ├── keycompute-db/                # Database ORM and migrations
│   ├── keycompute-auth/              # Auth & authorization (JWT + API Key + Password)
│   ├── keycompute-cache/             # Cache abstraction and Redis support
│   ├── keycompute-ratelimit/         # Rate limiting engine (In-memory/Redis dual backend)
│   ├── keycompute-pricing/           # Pricing engine (Three-tier + LRU cache)
│   ├── keycompute-routing/           # Two-layer smart routing engine
│   ├── keycompute-runtime/           # Runtime (AES-256-GCM encryption + storage abstraction)
│   ├── keycompute-billing/           # Billing & settlement (Post-stream precise settlement)
│   ├── keycompute-distribution/      # Referral distribution system
│   ├── keycompute-observability/     # Observability three pillars
│   ├── keycompute-config/            # Separate debug-TOML and release-environment loaders
│   ├── keycompute-emailserver/       # SMTP email service
│   ├── keycompute-payment/           # Payment integration
│   │   ├── keycompute-alipay/        # Alipay payment
│   │   └── keycompute-wechatpay/     # WeChat Pay
│   ├── llm-gateway/                  # LLM execution gateway (single upstream layer)
│   ├── llm-protocol/                 # Protocol translation and provider definitions
│   │   ├── openai/                   # OpenAI-compatible protocol
│   │   ├── anthropic/                # Anthropic-compatible protocol
│   │   └── provider/                 # Shared provider protocol types
│   ├── node-gateway/                 # Node gateway (registration/heartbeat/task management)
│   └── integration-tests/           # End-to-end integration tests
├── packages/                         # Frontend (Dioxus 0.7.9)
│   ├── web/                          # Web admin dashboard
│   ├── ui/                           # Shared UI component library
│   ├── desktop/                      # Desktop native application
│   ├── mobile/                       # Mobile cross-platform application
│   └── client-api/                   # API client wrapper
├── nginx/                            # Nginx reverse proxy configuration
├── Dockerfile.server                 # Backend container image
├── Dockerfile.web                    # Frontend container image
├── docker-compose.yml                # Production container orchestration
├── docker-compose.dev.yml            # Host-local dependency port overrides
└── docker-compose.replicas.yml       # Production read-replica orchestration

Configuration

Environment variables

This table applies to release/Docker production startup. A debug cargo run uses config.toml instead.

Variable	Description	Required
`KC__DATABASE__URL`	PostgreSQL connection string	✅
`KC__AUTH__JWT_SECRET`	Production: non-default/non-blank JWT signing secret, at least 32 bytes	✅
`KC__CRYPTO__SECRET_KEY`	Production: 32-byte Base64 KEK for Provider encryption and per-record Produce AI Key envelope encryption; rotate only after re-encrypting/rewrapping stored records	✅
`KC__NODE_GATEWAY__REGISTRATION_TOKEN_SECRET`	Production with Redis: non-default HMAC signing secret of at least 16 bytes; issues one-time node registration tokens	Conditional
`KC__REDIS__URL`	Redis connection string (optional; without it: rate limiter falls back to in-memory, cache no-ops, node gateway unavailable)	⚪
`KC__EMAIL__SMTP_HOST`	SMTP host (optional)	⚪
`KC__EMAIL__SMTP_PORT`	SMTP port (optional)	⚪
`KC__EMAIL__SMTP_USERNAME`	SMTP username (optional)	⚪
`KC__EMAIL__SMTP_PASSWORD`	SMTP password (optional)	⚪
`KC__EMAIL__FROM_ADDRESS`	Sender email address (optional)	⚪
`KC__EMAIL__FROM_NAME`	Sender display name (optional)	⚪
`KC__EMAIL__REQUIREMENT_RECIPIENT`	Requirement collection recipient email (optional; required to receive homepage submissions)	⚪
`APP_BASE_URL`	Current deployment's public frontend URL; required when SMTP is enabled and before enabling public invite links	Conditional
`KC__DEFAULT_ADMIN_EMAIL`	Default administrator email (optional)	⚪
`KC__DEFAULT_ADMIN_PASSWORD`	One-time bootstrap password: required only when production creates the first platform root identity; non-default/non-blank and at least 12 characters	Conditional

API

OpenAI-compatible API

# Chat Completions (streaming + non-streaming)
curl http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer sk-xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-chat",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

# List available models (defaults to the openai ingress protocol)
# Pass ?protocol=anthropic to list anthropic-protocol models instead
curl http://localhost:3000/v1/models \
  -H "Authorization: Bearer sk-xxx"

Account-to-tenant passthrough

Use Upstream Channels → Account Management / Passthrough Bindings / Node Gateway for upstream configuration. A passthrough binding grants one tenant access to all models declared on one account. There is no per-model binding or enabled switch. Global access and account-pool participation both default to false: only the selected tenant can use /pt, and ordinary account-pool or existing Responses resource paths cannot bypass the grant.

curl http://localhost:3000/pt/v1/chat/completions \
  -H "Authorization: Bearer $PLATFORM_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"example-model","messages":[{"role":"user","content":"Hello!"}]}'

curl http://localhost:3000/pt/v1/models \
  -H "Authorization: Bearer $PLATFORM_KEY"

Requests make at most one upstream generation attempt, without retries or fallback. Saving a grant requires no per-model activation and makes no paid probe call. Optional diagnostics are explicit. See docs/passthrough-bindings.md for scope flags, revocation, health, API contracts and fresh-schema requirements, and docs/upstream-channels.md for the administration flow.

The Responses API is available through POST /v1/responses and a WebSocket upgrade on GET /v1/responses. KeyCompute's WebSocket mode currently accepts response.create events only; connection-control events such as response.steer and response.inject are not supported.

Console API Overview

Category	Endpoint	Description
Auth	`POST /api/v1/auth/register`	User registration
	`POST /api/v1/auth/login`	User login
	`POST /api/v1/auth/forgot-password`	Forgot password
User	`GET /api/v1/me`	Current user info
	`GET/POST /api/v1/keys`	API Key management
Billing	`GET /api/v1/usage`	Usage statistics
	`GET /api/v1/billing/records`	Billing records
Payment	`POST /api/v1/payments/orders`	Create payment order
	`GET /api/v1/payments/balance`	Balance inquiry
Distribution	`GET /api/v1/me/distribution/earnings`	Distribution earnings
Node	`GET /api/v1/me/node-gateway/token`	Node token
	`GET /api/v1/me/tips`	Tip summary
Admin	`GET/POST /api/v1/platform/accounts`	Upstream account management
	`GET/POST /api/v1/admin/passthrough-bindings`	List/create account-to-tenant grants
	`PUT/DELETE /api/v1/admin/passthrough-bindings/{id}`	Update/delete with optimistic revisions
	`POST /api/v1/admin/passthrough-bindings/{id}/probe`	Optional single-model diagnostic
	`GET/POST /api/v1/platform/settings`	System settings
	`GET/POST /api/v1/platform/pricing`	Pricing management
	`GET /api/v1/platform/monitoring/overview`	Monitoring overview

For complete API documentation, refer to the route definitions in the project source code.

Development Guide

# Build (exclude desktop and mobile)
cargo build --workspace --exclude desktop --exclude mobile --verbose

# Run unit tests
cargo test --lib --workspace --exclude desktop --exclude mobile --verbose

# Run integration tests
cargo test --package integration-tests --tests --verbose

# Run frontend API client tests
cargo test --package client-api --tests --verbose

# Clippy code checks
cargo clippy --workspace --exclude desktop --exclude mobile --all-targets --all-features --future-incompat-report -- -D warnings

# Code formatting check
cargo fmt --all --check

# Release build
cargo build -p keycompute-server --release

Contributing

We welcome contributions of all kinds. Please read CONTRIBUTING.md to learn how to get involved.

License

This project is open sourced under the MIT License.

💖 Thanks for using KeyCompute

If this project helps you, feel free to give it a ⭐️ star.

Quick Start • Report Issues • Latest Releases

Native protocols and managed Responses

Account-pool (/v1), passthrough (/pt/v1) and NodeDispatch (/nt/v1) remain distinct execution families. Scoped Chat, Messages and Responses support complete native payloads and operation-specific SSE. KeyCompute can additionally manage scoped Responses history, Conversations, background work and cancellation without requiring stateful Ollama support. See native streaming and platform Responses state for API operations, capability negotiation, scope boundaries, retention and limits.

Name		Name	Last commit message	Last commit date
.github		.github
crates		crates
docs		docs
nginx		nginx
packages		packages
prod-build/web		prod-build/web
scripts		scripts
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
.review_prompt.md		.review_prompt.md
AGENTS.md		AGENTS.md
CONTRIBUTING.md		CONTRIBUTING.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
Dockerfile.server		Dockerfile.server
Dockerfile.web		Dockerfile.web
LICENSE		LICENSE
README.ar.md		README.ar.md
README.es.md		README.es.md
README.md		README.md
README.zh-CN.md		README.zh-CN.md
README.zh-TW.md		README.zh-TW.md
clippy.toml		clippy.toml
config.example.toml		config.example.toml
docker-compose.dev.yml		docker-compose.dev.yml
docker-compose.proxy.yml		docker-compose.proxy.yml
docker-compose.replicas.yml		docker-compose.replicas.yml
docker-compose.yml		docker-compose.yml
logo.jpg		logo.jpg

Navigation Menu

Repository navigation

About

Repository files navigation

KeyCompute

Overview

Features

Compute Node Leasing

Unified Multi-model Gateway

Smart Routing Engine

Billing & Payment System

Referral Distribution System

Authentication & Permissions

Observability

Cross-platform Frontend

Architecture

Quick Start

Requirements

Option 1: Docker Compose deployment (recommended)

Option 2: Local development

Project Structure

Configuration

Environment variables

API

OpenAI-compatible API

Account-to-tenant passthrough

Console API Overview

Development Guide

Contributing

License

💖 Thanks for using KeyCompute

Native protocols and managed Responses

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

KeyCompute

Overview

Features

Compute Node Leasing

Unified Multi-model Gateway

Smart Routing Engine

Billing & Payment System

Referral Distribution System

Authentication & Permissions

Observability

Cross-platform Frontend

Architecture

Quick Start

Requirements

Option 1: Docker Compose deployment (recommended)

Option 2: Local development

Project Structure

Configuration

Environment variables

API

OpenAI-compatible API

Account-to-tenant passthrough

Console API Overview

Development Guide

Contributing

License

💖 Thanks for using KeyCompute

Native protocols and managed Responses

About

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages