Skip to content

Releases

b12464

b12464 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 06:28
d3c3e24

b12463

b12463 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 06:14
9581154

b12458

b12458 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 05:36
92192d3

mtmd: support YuE2-3B (#32369)

  • mtmd: support YuE2-3B

  • improve speed

  • add CFG

  • fix streaming

  • add cfg control from api

  • check ctx size

  • fix n_predict

  • add flow_steps config

  • nits

  • fix batch api

  • nits

  • adapt

  • mtmd: yue2 samples from its own saved logits

The context logits belong to the last decode of any slot, so with
several server slots the score and the unguided codes were sampled
from another sequence. Keep the logits of each own decode in
save_logits(), which also does the guidance mix.

  • mtmd: let the yue2 snake chain fuse

The beta reshape landed between sqr and mul, so the mul, sin, sqr,
mul, add chain was not consecutive and the backends never fused it.
Expand both reshapes first, like qwen3tts does.


Co-authored-by: Pascal [email protected]

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b12457

b12457 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 05:20
ad5cbfa

b12456

b12456 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 05:05
d261e49

opencl: fix three defects of the bin flash-attention prefill path (#32110)

  • opencl: release the WMM flash-attention mask fallback image

The fallback image is created on every call of the bin flash-attention
kernel over a sub-buffer of the K cache and was never released, so each
call kept a K-cache sub-buffer alive. Every call with more than one query
row takes this path, including each speculative-decoding verify step, so
the objects pile up and the server slows with every request.

  • opencl: keep q8_0 K/V at head size 512 off the WMM flash-attention kernel

With q8_0 K/V at DK=512 the bin flash-attention kernel gives wrong results
whenever V holds different values from K (test-backend-ops FLASH_ATTN_EXT
hsk=512 q8_0, max error 2.04 on X2-90); f16 K/V at DK=512 is correct. The
backend's own flash-attention kernels already decline q8_0 at DK=512, so the
shape is now handled the same way with or without the kernel library.

  • opencl: decline WMM flash-attention prefill beyond the device's 3D image limits

The WMM path repacks Q/K/V into 3D images; V is n_kv wide, so past the device's
maximum 3D image width (about 16k tokens of context on X2-90) clCreateImage
failed and the call asserted. Such shapes now take the regular flash-attention
path.

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b12455

b12455 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 04:48
4ddf2b9

b12453

b12453 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 04:16
c7518db

b12452

b12452 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 03:59
a68c613

b12451

b12451 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 03:34
3a76603

ui: move the web UI to ui/apps/chat and call it Llama Chat (#32032)

  • ui: move tools/ui to ui/apps/chat

The chat app moves to its own root folder, ui/apps/chat, as the first step of splitting the UI into one library
(ui/lib) and the apps built on it (ui/apps). The code is unchanged; every path that pointed at tools/ui now points at
ui/apps/chat: CMake adds it from tools/, and the Dockerfiles, the Nix package, the workflows, the labeler, CODEOWNERS,
.editorconfig, the git hooks, the dev and demo scripts and the docs follow it.

Assisted-by: Claude (Opus 5.5 Medium)

  • ui: name Llama Chat's package @ggml-org/llama-chat

It was still called llama-ui, the name of the library that now sits beside it
in ui/lib (@ggml-org/llama-ui). The package stays private; the app's display name
and the llama-ui.zip build artifact are unchanged.

Assisted-by: Claude (Opus 5.5 Medium)

  • ui: call the built-in web UI Llama Chat in docs, help text and comments

The app in tools/ui is Llama Chat; Llama UI is the component library it is
built from. Prose that said "Web UI", "web UI" or "WebUI" for the app now
names it: the READMEs, SECURITY.md, the --ui help text, the CMake option
description, one server log line, code comments and the Storybook intro.

Names that others depend on are unchanged: the deprecated --webui flag
aliases, the LLAMA_*_WEBUI CMake and environment names, the webui_settings
field, the storage migration keys and the Nix attributes.

Assisted-by: Claude (Opus 5.5 Medium)

  • ui: name tools/ui as a former path of Llama Chat in the code-review skill

Changes under the old tools/ui path, as well as tools/server/webui, still
get the Llama Chat checklist.

Assisted-by: Claude (Opus 5.5 Medium)

  • ui: name Llama Chat's path in the security policy's exclusions

Llama Chat no longer lives under tools/server/, so the exclusion listed
under it now names ui/apps/chat. What is covered stays the same.

Assisted-by: Claude (Opus 5.5 Medium)

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

b12449

b12449 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 17 Nov 03:13
7804be0