中文版: README.zh-CN.md
Web client for the Myriad-MoE 25,200 micro-expert model, talking to an OpenAI-compatible inference service (reference implementation: 7.api_myriad_server.py).
- Left: ChatGPT-style conversation area with foldable chain-of-thought, slash commands, regeneration
- Right: holographic neural introspection board — arts/sci core split, cluster heatmap, cage / single-layer snipe, elastic Top-K, per-layer radar, cartridge hot-plug
- Language: 中文 / English toggle in the top bar (persisted in
localStorage)
This repo contains only the frontend. The backend inference service and model weights (
myriad_*.pt, see 🤗 Hugging Face) live elsewhere and must be deployed separately.
In the server repo root:
uv pip install --python .venv/bin/python fastapi "uvicorn[standard]"
.venv/bin/python 7.api_myriad_server.py \
--host 0.0.0.0 --port 8000 \
--weights myriad_moe_hierarchical_weights.pt \
--backend-script 6.chat_myriad_25k_lora_fast_more_mirco.pyModel loading takes a while. During that window:
GET /health→{"status": "loading", "ready": false}GET /v1/models→ 200, butdata[0].myriad.layersisnull(the client uses this to detect "loading")GET /v1/chat/completions→ 503 +startup_error- All
GET/POST /v1/myriad/*→ 503 +model is still loading ...
The top badge shows Loading model (blue spinner) instead of a red error, then flips to "Online" automatically once loading finishes and the board starts refreshing.
Optional auth: pass --api-key sk-xxx, after which every request needs Authorization: Bearer sk-xxx.
npm install
npm run devClick Settings (top right) and confirm API Base URL is:
http://127.0.0.1:8000/v1
The /v1 prefix is mandatory — every business endpoint lives under /v1. A missing prefix yields 404, which the client explains explicitly.
Fill in the API Key (if the backend has auth enabled) and hit Test.
No GPU? You can still explore the full console UI:
- Settings → check Offline mock fallback
- The top badge switches to "Mock demo", and both chat and board show a prominent mock data banner
This mode is off by default and only kicks in when the service is completely unreachable. These real errors are never mocked and always surface as-is:
| Situation | HTTP | Client behavior |
|---|---|---|
| Wrong API key | 401 | Error, points you to Settings to check the key |
Base URL missing /v1 |
404 | Error, explicitly mentions the /v1 prefix |
| Model still loading | 503 | Error, shows the server's startup_error |
| Illegal params (layer/cluster out of range, …) | 400 | Error, shows the server detail |
| Generation thread crash (CUDA OOM, …) | SSE error frame |
Error, never disguised as an empty answer |
This is deliberate — a misconfiguration should never look like "the model is running fine".
The server is stateless; the client owns the conversation history and resends it every turn.
| Command | Effect |
|---|---|
/help |
Capability cheat sheet |
/clear |
Clear local context (actually empties messages) |
/stats |
Arts/sci split + top clusters + CUDA Graph status |
/clusters |
Hit counts and cage state of the 20 clusters |
/show_k |
Per-layer Top-K |
/set_k <layer> <k> |
Tune one layer |
/set_k_all <k> |
Tune all layers |
/cage <cluster> |
Global cage (zero all layers' weights of that cluster + chill its routing) |
/free <cluster> |
Release (always releases the whole cluster, see below) |
/snipe <layer> <cluster> |
Single-layer snipe |
/plug <cartridge.pt> [slot] |
Cartridge hot-plug |
/graph |
Toggle CUDA Graph |
/maxlen <n> |
Change default reply length |
The server keeps both in the same state table (_caged[cid] is a dict keyed by layer):
cage(cid)→ writes all 28 layers = global cagesnipe(layer, cid)→ writes 1 layer = single-layer snipefree(cid)→ restores all recorded layers of that cluster, then drops the entryfree(cid, layer)→ restores only that layer; the entry is dropped only once no caged layer remains
Both release granularities are available:
| Action | Endpoint | Effect |
|---|---|---|
| Release all | POST /v1/myriad/clusters/{cid}/free |
Restore every caged layer of that cluster |
| Free one layer | POST /v1/myriad/clusters/{cid}/free?layer=7 |
Restore only layer 7 |
Each L07 ✕ button in the cage panel is a single-layer free; the "Release all" button frees the whole cluster. Caged layers are shown via the server's caged_layer_map field; fully_released / still_caged_layers in the response tell whether the release is complete.
The server supports two token kinds:
| Token | Flag | Permissions |
|---|---|---|
| Admin | --api-key |
Everything: telemetry + neural surgery + tuning + hot-plug + generation |
| Read-only | --read-only-key |
Only GET telemetry/board; every write returns 403 |
python 7.api_myriad_server.py --api-key sk-admin --read-only-key sk-viewerGET /v1/models reports the current token tier in the myriad field, and the client accordingly:
- Shows a Read-only badge + purple banner explaining the limits
- Disables all write buttons (cage / release / snipe / single-layer free / tuning / hot-plug / stats reset / CUDA Graph toggle)
- Blocks message sending to avoid doomed 403 requests
- The Settings "Test" button directly reports whether the current token is read-only or admin
When the server has no key configured, the UI shows a No auth badge as a reminder that anyone can operate it.
Hot-plug in the cage panel supports two sources:
- Local file upload — pick a
.ptfile, uploaded via multipart with a progress bar - Server path — a filename under the server process working directory
Uploads are capped by --max-cartridge-mb (default 512 MB); over-limit uploads return 413 with a readable message. The filename is display/record only and never used for path joining — the server sanitizes it with basename.
The server is stateless, so the client resends history every turn. To keep prompts from growing forever, history is trimmed before sending to:
[system prompt (always kept)] + latest N rounds
One round = one user message + the assistant messages that follow it. N defaults to 10, adjustable 4–30 in Settings.
Also stripped automatically: local notes (error echoes, /clear confirmations), empty assistant messages (leftovers from interrupted generation — some chat templates choke on them), and in-flight streaming placeholders.
When trimming happens, a Context 10/10 rounds · trimmed N note appears under the input box.
The client always sends myriad: { stats: true }, and the server attaches request-scoped telemetry in the last frame of the stream (before [DONE]):
{"choices": [], "usage": {...},
"myriad": {"arts_core_pct": 62.0, "sci_core_pct": 38.0,
"cuda_graph": true, "top_clusters": [...]}}Every answer in the chat area carries a telemetry strip at the bottom: arts/sci split bar, CUDA Graph state, dominant clusters (caged ones struck through).
| Field | Default | Notes |
|---|---|---|
baseUrl |
http://127.0.0.1:8000/v1 |
Must include /v1 |
apiKey |
empty | Required when backend --api-key is on |
model |
myriad-moe-25k-lora |
Matches server SERVED_MODEL_ID |
systemPrompt |
empty | Always first in context; empty = no system message sent |
contextRounds |
10 | Context rounds to keep (4–30) |
temperature |
0.7 | |
topP |
0.9 | |
maxTokens |
1024 | Server clamps to [1, 8192] |
repetitionPenalty |
1.15 | Server native default |
splitReasoning |
true | Split <think> into reasoning_content |
pollIntervalMs |
5000 | Board poll interval; auto-pauses while generating |
mockFallback |
false | See above |
focusClusters |
[] |
Non-empty: unlisted clusters are chilled for this request |
topKOverride |
null |
Per-request Top-K override |
Settings and chat history are stored in localStorage.
The top bar has a 中/EN toggle; the choice persists in localStorage (myriad_lang, defaults to the browser language). It covers all UI chrome, banners, toasts and error hints. Model-generated content itself is never translated.
- Single-slot decoding: the server handles one generation request at a time; concurrent requests queue (
waiting_requestsvisible on the board). - Board polling costs GPU:
/v1/myriad/statsdoes 28 synchronous.cpu()copies server-side. So the client pauses polling while generating, default interval 5 s. Polling too aggressively slows down token streaming. - Streaming telemetry also runs one
dashboard(): the telemetry frame at the end of each answer costs one extra stats snapshot per request (cheaper than the old 2 s polling). slot_statemay readbusy: it is sampled before the response closes, while the generation thread hasn't released its semaphore yet. The next board poll self-corrects.127.0.0.1means the browser's machine: when opening the UI from another computer, point Base URL at the backend's real IP.- Only the latest N rounds are visible to the model: limited by the context sliding window.
npm run typecheck # tsc --noEmit
npm run verify # typecheck + all frontend logic tests
npm run build # output in dist/
npm run preview| Command | Covers | Needs backend |
|---|---|---|
npm run verify:client |
Error routing, SSE parsing, math rendering, context window | No |
npm run verify:server |
API contract (routes / auth / CORS / error codes / streaming telemetry), loading-phase behavior, polling policy | Yes |
verify:server needs 7.api_myriad_server.py. It looks in the parent directory by default, or take a path from the environment:
MYRIAD_SERVER=/path/to/7.api_myriad_server.py npm run verify:serverIt skips gracefully with an explanation (exit 0) when not found, so npm run verify won't fail on a standalone clone. It also needs fastapi and torch — run it in a Python env with the backend deps installed.
Stack: React 19 + TypeScript + Vite 8 + Tailwind CSS 4. KaTeX and fonts are local dependencies, no CDN required (works on intranets).
This project is Apache-2.0, matching the backend Myriad-MoE-25K-Micro-Experts. Academic and commercial use allowed; please keep the original attribution when reposting or deriving.
@misc{feifei2026myriadconsole,
author = {FeiFei (aifeifei798)},
title = {{Myriad-MoE Neural Console: A Web Client for the 25,200 Micro-Expert Dual-Core MoE}},
year = {2026},
publisher = {GitHub},
howpublished = {\url{https://github.com/aifeifei798/myriad-moe-console}}
}