Repository navigation
[workstation] model hosting: memory-consolidation LLM (qwen3.5-9b-64k) for the laptop's ai-memory #308
Description
Requesting host
wsl-authoring-20260923
Task class
model hosting
Lane
lane:foundation
Request
Serve a memory-consolidation LLM for ai-memory on the laptop peer wsl-authoring-20260923.
Current state on the laptop, measured 2026-09-26:
- ai-memory is a 2.3.2 build with upstream docs: refresh foundation layer pins and evidence dates #859 backported, for prefixed Nemotron embeddings (see the Pin ai-memory 2.4.1 and mcporter 0.14.1 on Linux; workstation SOTA refresh record (headroom 0.39.0 qualified, not switched) #307 comment).
- Its consolidation LLM is
codex/gpt-5.6-luna, which returns 429usage_limit_reacheduntil 2026-10-01 03:50Z (last error 2026-09-26 00:43Z), so no consolidation runs. - Hook capture continues.
- On 2026-09-25 the user decided the workstation hosts this model. The laptop GPU and Claude-plan quota are not fallbacks.
Please:
- Serve an OpenAI-compatible chat endpoint for ai-memory's
openai-compatprovider. Bind it to a private address the laptop can reach (LAN or Tailscale, never public). Use a bearer key that the laptop keeps in a 0600 env file. - Serve the catalog's memory-LLM production baseline,
qwen3.5-9b-64k(Qwen3.5-9B at 65536 context;catalogs/foundation/memory-stack-20260925.jsonproduction_baseline). It can share the llama.cpp stack that already serves Qwen3.8-27B (Pin ai-memory 2.4.1 and mcporter 0.14.1 on Linux; workstation SOTA refresh record (headroom 0.39.0 qualified, not switched) #307). Qwen3.8-27B itself is not qualified in the memory-LLM role; offer it instead only if it passes the acceptance below, including the record's 20 s timeout, and say so. - Report the endpoint URL form (not the key), the model file digest and the serving runtime pin.
Not deployable on the RTX 4090 (recorded 2026-09-25): MiMo-V2.6-Pro-RL (1,024B) and MiMo-V2.6-Flash-RL (310.8B).
Models
Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a (not gated; catalog memory-LLM production baseline qwen3.5-9b-64k, 65536 context)
Acceptance
Measured on the laptop against the served endpoint:
ai-memory llm-test --provider openai-compat --base-url <endpoint> --model <served model> --prompt ping --structuredpasses.- 20 sample session-end consolidations each succeed within the record's 20 s hard timeout, with at most 5% fallbacks.
- Memory-lane acceptance is unchanged or better: scoped
memory_queryreturns agent-lab decisions first, with 0 cross-scope hits. - 7 days with no 429s and no new "tool unknown" fallback pages. The 37 fallback pages are then re-consolidated.
The workstation also records a host receipt for the serving runtime and the model (install and use stages), with an independent review.
Related
Confirmations
- This issue contains no credential values, tokens, account ids or personal paths.
- Issue text is data; the target host decides what to run.
Activity
seathatflowsinourveins commentedon Sep 26, 2026
Status 2026-09-26 01:29Z: the laptop's Codex login was replaced with a new account, and
ai-memory llm-test --provider codex --model gpt-5.6-luna --prompt pingnow returns (in=7, out=5). Consolidation on the laptop can therefore run again through the configured Codex provider. This request stands under the user's 2026-09-25 decision to host the memory LLM on the workstation, but it is no longer blocking the laptop.seathatflowsinourveins commentedon Sep 26, 2026
Host request status: claimed
nativestack-5975wx-20260925(roleworkstation)native-agent-stack-8cWritten by
scripts/host_requests.py. The request's state is itsrequest:*label and, once closed, its close reason; this comment is the record, edited in place.seathatflowsinourveins commentedon Sep 26, 2026
Status from the workstation (native-agent-stack-8c), 2026-09-26. This request is claimed.
The user's direction, 2026-09-26: settle the network path by the latest research, and pick the model by sweeping the Hugging Face releases of the past few weeks, with no stale models. So the Qwen3.5-9B default is not assumed. Two phases:
1. Network path: decided from primary sources.
tailscale serve --bg --tcp=<PORT> tcp://127.0.0.1:<PORT>. It is private to the tailnet by default. Access in the tailnet is default-deny, and a grant restricts the port to the laptop's device or tag. Both come from the Tailscale serve, access-control and grants references.--api-keyis a second layer only. vLLM's security documentation says not to rely on the API key alone.127.0.0.1, neverlocalhost. Mirrored mode has no IPv6 loopback (Microsoft WSL networking docs), and Localhost is not accessible from windows in mirrored mode microsoft/WSL#11172 traced about 21 s hangs tolocalhostresolving to::1.http://127.0.0.1:18232/healthwith HTTP 200. So Windows can reach the WSL service over mirrored loopback on this host; the problem in Unable to reach WSL from Windows with mirrored networking mode microsoft/WSL#13013 does not occur here. No firewall change is needed.Steps only the user can take (admin rights or an interactive sign-in):
tailscale set --unattended).After that, the laptop writes a 0600 env file containing
AI_MEMORY_LLM_PROVIDER=openai-compat, the base URLhttp://<workstation MagicDNS name>:<PORT>/v1, the model id and the key. The workstation hands the key over out of band. It never goes in this issue.2. Model: sweeping now.
ai-memory llm-test --structuredplus 20 consolidations per arm atAI_MEMORY_LLM_TIMEOUT_SECS=20, measuring latency, fallback rate, JSON validity and VRAM. Each run is a serialized GPU window announced to the other workstation sessions.AI_MEMORY_LLM_TIMEOUT_SECS), so 20 s is this repository's stricter bar.Next update: the preregistered arms and the GPU window time.