parakeet.cpp has an opt-in offline beam decoder for TDT checkpoints. It
returns multiple complete transcript hypotheses without changing the existing
greedy decoder.
pk::tdt_beam_search follows NVIDIA NeMo 2.7.3's
BeamTDTInfer.default_beam_search:
- Apply
log_softmaxindependently to token and duration logits. - Expand the highest-scoring non-blank token-duration pairs.
- Expand blank only with non-zero durations.
- Merge paths with the same token sequence and final frame using
logaddexp. - Sort by
score / (emitted_tokens + 1)by default. The extra item is NeMo's leading blank/SOS sentinel.
The C++ loop additionally rejects non-finite log probabilities and a zero-duration expansion that does not strictly reduce the accumulated score. This prevents malformed logits or floating-point saturation from creating a non-progress loop without truncating valid hypotheses.
There is one intentional edge-case difference from NeMo's reference loop.
When the duration beam contains only duration 0, blank cannot use that
duration. NeMo substitutes the smallest positive duration; this implementation
substitutes the highest-scoring positive duration so the legal blank expansion
preserves model score ordering. A model-independent regression covers this
case for beam_size=1.
The implementation is limited to the default sequence-level TDT beam search.
It does not add mAES/mALSD, an external language model, batched beam search, or
streaming beam search. NeMo labels this strategy experimental and recommends
malsd_batch;
this first implementation deliberately prioritizes a small, directly
comparable reference algorithm over a second decoder design.
parakeet-cli transcribe \
--model parakeet-tdt-0.6b-v3-q8_0.gguf \
--input audio.wav \
--decoder tdt \
--beam-size 4 \
--nbest 4The command emits JSON:
{
"beam_size": 4,
"score_norm": true,
"frame_sec": 0.08,
"hypotheses": [
{
"text": "A complete transcript.",
"score": -0.941021,
"normalized_score": -0.020912,
"tokens": [
{
"id": 499,
"frame": 4,
"t": 0.32,
"duration_frames": 1,
"duration": 0.08
}
]
}
]
}Use --no-score-norm to rank by raw accumulated score. --nbest defaults to
the beam size.
The additive entry points are:
parakeet_capi_transcribe_path_nbest_json(...);
parakeet_capi_transcribe_pcm_nbest_json(...);Both reuse a load-once parakeet_ctx, accept an optional language prompt, and
return the same JSON document as the CLI. Existing symbols and ABI version 5
are unchanged.
Generate an authoritative NeMo baseline:
.venv/bin/python scripts/gen_nemo_baseline.py \
--model nvidia/parakeet-tdt_ctc-110m \
--audio tests/fixtures/speech.wav \
--tdt-beam-size 4 \
--output /tmp/tdt_nbest_baseline.ggufThen compare token IDs, token end frames, rank order, and raw scores:
PARAKEET_TEST_GGUF=/tmp/pk110m.gguf \
PARAKEET_TEST_TDT_NBEST_BASELINE=/tmp/tdt_nbest_baseline.gguf \
ctest --test-dir build --output-on-failure -R '^test_tdt_beam$'The pull-request closed-loop job generates this baseline directly with NeMo and runs the comparison.