Fix Linux support: by-value struct ABI, pinned llama.cpp b11146, stack-safe C buffers - #2

Summary

Janus currently builds on Linux but cannot load a model there. This PR makes the Vulkan runner work on Linux/amd64 and keeps the Windows path compatible. Tested end to end on an NVIDIA RTX 5060 Ti (16 GB) via Vulkan.

What was broken on Linux

1. Struct-by-value ABI. llama_load_model_from_file, llama_new_context_with_model and llama_decode take their params/batch structs by value. Janus passed a pointer (uintptr). That happens to match the Windows x64 ABI (large structs go by hidden pointer) but not SysV amd64, so on Linux model/context creation failed (n_batch and n_ubatch cannot both be zero, NULL context).
2. build.sh download. The llama.cpp "latest" release is now v0.5.0, which has no binaries. Binary releases use bNNNNN tags and ship .tar.gz (not .zip) on Linux.
3. Backend loading. ggml_backend_load_all was only looked up in ggml.dll, so no backend (including Vulkan) registered on Linux.
4. JANUS_GPU_LAYERS=-1 depended on the library default. -1 meant "keep the lib default", but that default changed across llama.cpp versions. With b5400 (default 0) the README config logged offloaded 0/29 layers to GPU and ran on CPU while reporting [vulkan].
5. Context params layout. The Go mirror of llama_context_params was written against an older llama.h, with n_threads at offset 16. Current llama.cpp (b11146, and also b11326/b11374, which build.ps1 was fetching) adds n_rs_seq, n_outputs_max and n_outputs_max_per_seq first, which moves n_threads to offset 28. So the thread count ended up in other fields.

Changes

• ABI (internal/bridge/abi_unix.go, abi_windows.go): non-Windows registers the three functions by value through purego and wraps them behind the existing pointer-taking fields. Windows keeps the direct pointer binding, which is exact for Win64. The three LlamaLib fields (LoadModelFromFile, NewContextWithModel, Decode) now take *LlamaModelParams / *LlamaContextParams / *LlamaBatch. The UnsafePtr helper and batchState.ptr() are removed because they no longer have callers. LlamaContextParamsSize goes from 512 to 192 bytes, which is purego's limit of 26 stack words on amd64; the real struct is 160 bytes.
• purego v0.8.2 → v0.11.1, needed for struct arguments on Linux. This raises the minimum to Go 1.25. The indirect golang.org/x/sys require is dropped (go mod tidy no longer needs it).
• Pinned llama.cpp b11146 in both build.sh and build.ps1 (override with --version / -LlamaVersion). The Go struct mirrors match that tag's llama.h. internal/bridge/headers/llama.h is replaced with the b11146 header, and llama_dl_test.go locks the offsets.
• Stack-moving pointer fix (shared code, so it should affect Windows too; only reproduced on Linux): buffers handed to C (the *_default_params sret buffers, TokenToPiece, Tokenize, ChatApplyTemplate) were passed as uintptr of goroutine-stack memory. The goroutine stack can move during the purego call, so C wrote to or read from stale memory. Reproduced against the b11146 lib: 150/200 calls from fresh goroutines returned n_batch=0. This hits /models/load (hot-swap runs on an HTTP goroutine). These are now unsafe.Pointer, and a regression test (llama_defaults_test.go, non-Windows) runs when the lib is present.
• Per-OS backend library names: libggml.so with libggml-base.so as fallback, libggml.dylib, ggml.dll.
• JANUS_GPU_LAYERS < 0 now maps to all layers. With the default config this should also make the existing context-OOM → CPU retry reachable (the nLayers > 0 guard); not exercised in testing.
• Download source: build.sh and build.ps1 now point at ggml-org/llama.cpp, where ggerganov/llama.cpp redirects.
• build.sh: works from any cwd, extracts .tar.gz, preserves .so symlinks (libllama needs libggml.so.0), replaces old libs only after the new ones are found, no longer runs go mod tidy, drops the LD_LIBRARY_PATH hint (the libs resolve through RUNPATH=$ORIGIN), and is now executable.
• Docs: README Linux quick start (the macOS notes are marked untested), Go 1.25+ in README and CONTRIBUTING.

Testing (Linux/amd64, RTX 5060 Ti, Vulkan, llama.cpp b11146)

• ./build.sh from a clean checkout, run from another cwd: OK, and the tracked tree stays clean.
• go vet ./... (linux and GOOS=windows), GOOS=windows GOARCH=amd64 go build ./..., go test -count=1 ./...: pass.
• Qwen2.5-7B-Instruct Q4_K_M with README defaults: offloaded 29/29 layers to GPU, n_threads=32. /health, /health?deep=true, /v1/models and /v1/chat/completions (normal and stream:true) all work. Hot-swap via /models/load creates the context with n_batch=2048.
• CPU fallback with JANUS_GPU_LAYERS=0: works.
• Throughput (7B, 200 tokens): ~50 tok/s median, vs llama-bench tg128 88.7 tok/s on the same lib. The gap predates this PR; it looks like per-token overhead in the generation loop, but I have not profiled it.

Not covered

• Windows runtime is not tested (no Windows host). Only go vet / go build for windows/amd64 were run. The purego bump and the unsafe.Pointer changes touch shared code.
• arm64/macOS: documented as unsupported. The *_default_params hidden return pointer goes in x8 on arm64, which this binding does not handle.
• Pre-existing gap: if llama_load_model_from_file returns NULL because VRAM runs out while loading weights, there is still no CPU fallback. Left for a follow-up.

🤖 Generated with Claude Code