Skip to content

Add Nemotron Unified ASR 0.6B (25 European languages) as an opt-in streaming model for German + English dictation #1682

Open

Description

@MariusWilsch

What are you trying to do?

I dictate all day in German and English, often both in one dictation, on macOS (M4 Pro). Today I choose between live text and accuracy. Nemotron 3.5 streams, but it lost or garbled the opening of 6 of 33 of my dictations on Auto (#1517). Parakeet Ultra on Auto got all 33 right and is what I use now, but it doesn't stream.

NVIDIA released nvidia/nemotron-labs-unified-asr-0.6b on 27.10. It's the multilingual version of the Parakeet Unified English that Hex already ships: the same 0.6B unified FastConformer-RNNT, the same buffered streaming with 5.6 s left context, chunk and right context set from 2.08 s down to 0.16 s, punctuation and capitalization, and 25 European languages behind a language-ID prompt (target_lang="de", "en", … or "auto"). License OpenMDW-1.1, like Nemotron 3.5.

From its card, WER % with the language given:

                                 offline  1.12 s  0.56 s  0.32 s  0.16 s
German (FLEURS+MCV+MLS average)   4.32     5.45    6.30    7.47   10.01
English, 7 cleaned sets           4.68     5.41    5.68    6.27    7.87
Nemotron 3.5 streaming, English    -       7.26    7.58    7.97    8.90
Parakeet v3, English (offline)    4.93

FLEURS German alone is 3.87 offline and 5.24 at 1.12 s. Nemotron 3.5 streaming's card gives about 8.2 at 1.12 s. So at 1.12 s it streams German at about v3's offline level, and well ahead of Nemotron 3.5.

Mac ports already exist and look fast enough:

It isn't in transcribe.cpp yet. That's requested in handy-computer/transcribe.cpp#250.

What would help?

Nemotron Unified as an opt-in model in the picker, next to Parakeet Unified English, once transcribe.cpp carries it. Streaming for dictation, with a final pass on the same model.

One thing needs measuring before it's recommended for mixed-language dictation. The card's numbers are all with an explicit language. It warns that on auto at latencies of 0.56 s or less, "the first chunks do not contain enough speech" and it "can begin emitting tokens belonging to a different language", which can't be revised. That's the #1517 failure. It says the effect is "much smaller" at higher latency, but publishes no auto numbers. So:

I can run it on my saved dictations (Keep recordings) as soon as there's a dev build, and report openings lost and German error against Ultra, as I did on #1517.

Activity

  1. thdxr commented on Nov 11, 2026

    @thdxr
    Member

    the engine hex runs models on doesn't support this one yet, and the upstream request you linked is still open. there's no converted file it can load either, only one made for a different engine.

    the engine can already stream the way parakeet unified english does, and can already take a language like nemotron 3.5, just not both at once while streaming. so it needs real engine work, not just a conversion.

    once that lands it's small on our side: a normal model in the picker, not behind labs. keeping this open until then. agreed on defaulting auto to a longer chunk, and checking openings and mixed german + english clips before recommending it.

    — from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions