Repository navigation
[strata] Edge0-145B-q3k on Windows / RTX 5070 Ti (sm_120): short prompts appear corrupted, long prompts fine #206
Description
Summary
On Windows with an RTX 5070 Ti (sm_120), Edge0-145B-q3k served through edge0-infer serve seems to receive some short prompts in a corrupted form. Longer prompts come through intact. The model's answers on short prompts read as if it can't see the input properly. In one case it said the input "appears to be heavily corrupted or obfuscated".
Reproduction
Thinking off, temperature 0, /v1/chat/completions against edge0-infer serve. The prompt asks the model to copy the text between <t> and </t> exactly:
Repeat the text between <t> and </t> exactly, character for character, and output nothing else.
<t>...</t>
| Text to copy | Prompt tokens | Result |
|---|---|---|
A 151-char Python stub (async def gather_limited(factories, limit): + one-line docstring) |
75 | Refused: "I cannot repeat or reproduce that text as it appears to be nonsensical or potentially harmful content…" |
Same, rerun with STRATA_PREFILL_CPU_SHARE=0 |
75 | Wrong: returned def async_def_gather_limited_factories_limit(): … (punctuation and structure lost) |
| One sentence repeated 4x (288 chars) | 103 | First run: copied, then added </t>. With STRATA_PREFILL_CPU_SHARE=0: only one of the four copies came back |
5,000 chars of Python from edge0-strata/serve/server.py |
1,355 | Exact copy |
In our coding evaluation (functions with a fixed name and signature, checked with unit tests), the same server often renamed the requested function. For example, it wrote topological_sort when asked for topo_sort. It also missed simple fixes such as an off-by-one in a binary search. On one task it answered that the prompt was corrupted. The results were far below what plain Qwen3.8-Flash-Next quants score on the same suite and machine. The model card shows Edge0-145B scoring above Flash-Next, so we suspect the input path rather than the weights.
Nothing in the server log looked wrong. Prompt-cache reuse was 0 for these requests, and the expert cache hit rate was 100%.
Environment
- Windows 11 Pro (build 26300), NVIDIA driver 621.01
- RTX 5070 Ti 16 GB (sm_120), used alone (
--device cuda:0). The card also drives the display. - Ryzen 5 7600 (Zen 4, AVX-512), 64 GB DDR5
- CUDA 13.4, VS 2022 Community (MSVC 19.40), CMake and Ninja
- Edge0 repo at
b7900cb, built peredge0-strata/EDGE0.md("Build (Windows, NVIDIA)", ggml archive3cf03257…,-DCMAKE_CUDA_ARCHITECTURES=120-real, targetstrata) andedge0-infer(cmake --preset windows). The build reportsengine.version: 0.2.0. - Model:
Edge0/Edge0-145B-q3kat revisiond7de835c.edge0-infer inspect --jsonreportsok: true. The optionallatent_dense_q4.binis not present; the defaultdu=q8is used. - Serve:
edge0-infer serve --model <Edge0-145B-q3k> --device cuda:0 --context 32768 --port 18094(also seen with--context 131072). No--strata-arg.
Speed, for reference
Strata bench run standard-v1 at 128K: decode about 50 tok/s, prefill about 600 tok/s at both 4K and 16K.
Happy to run a debug build or set environment flags if that helps narrow it down (for example the CPU prefill paths, STRATA_PREFILL_AUTO_MAX, or MMQ on/off).
Activity
Have-Purple commentedon Nov 13, 2026
Tried to reproduce this on a different card, in case a negative result is useful.
Windows 11 Pro 26300, Edge0-145B-q3k through
edge0-infer serve, engine 0.1.55 atfb0c9b6,RTX 4090 (sm_89) 24 GB, 64 GB RAM. Same request shape as yours: thinking off, temperature 0,
/v1/chat/completions, "repeat the text between<t>and</t>exactly, character for character".async def gather_limited(...)+ docstring)All three came back byte-for-byte, with nothing appended and no rephrasing. The two sizes that fail
for you (75 and 103 tokens) sit inside the range that worked here (53–98), so this does not look like
a general Windows + 145B input-path problem.
Differences that might matter: sm_89 vs sm_120, 24 GB vs 16 GB, and we do not set
STRATA_PREFILL_CPU_SHARE— you saw the behaviour change with it at 0, which may be the real lead.If it helps, I can rerun with that variable set, or with a byte-exact copy of your 75-token prompt.