Add parakeet_capi_transcribe_pcm_logits: expose CTC log-probs by aanchan · Pull Request #1 · happy-prime-inc/parakeet.cpp

aanchan · 2026-07-28T01:55:15Z

Summary

classroom-captions#63: the C-API only ever returns decoded text/timestamps/JSON,
even though CTCDecoder::forward already computes a log-softmaxed [T, vocab+1]
matrix internally. We need that matrix directly so classroom-captions's
existing pyctcdecode + KenLM + hotwords decode stack can run on top of this
engine (the same seam it already uses for the CoreML backend).

What's included

Model::transcribe_pcm_ctc_logits / transcribe_16k_ctc_logits (src/model.hpp,
src/model.cpp): mirrors transcribe_pcm/transcribe_16k through mel + encoder
(including the existing long-audio tiling path), then runs only the CTC
head, skipping greedy/beam decode. Throws std::runtime_error if the model
has no CTC head (e.g. a pure TDT/RNNT streaming model).
parakeet_capi_transcribe_pcm_logits / parakeet_capi_free_logits
(include/parakeet_capi.h, src/parakeet_capi.cpp), following the existing
validate -> try/catch -> last_error conventions. ABI bumped to v6.
tests/test_capi_ctc_logits.cpp, two independent blocks: self-consistency
against the real parakeet-ctc-1.1b checkpoint (argmax-greedy over the
exposed matrix reproduces transcribe_pcm(..., kCTC)'s own greedy text
byte-for-byte), and error-path coverage against a no-CTC-head model
(clean failure, no crash).

Verified end-to-end through classroom-captions's Python ctypes binding too:
WER parity vs. the existing CoreML backend on real multi-accent fixtures
(20-45s clips) shows this backend beating CoreML ~28% relative WER (greedy
0.137 vs 0.195, LM 0.141 vs 0.190), with none of the chunk-boundary
duplication artifacts CoreML's fixed 5s window produces on longer audio.

Next step

Not yet offered upstream to mudler/parakeet.cpp — landing here first.

Copilot

Pull request overview

This PR adds a new logits-oriented seam to parakeet.cpp’s public C-API (and underlying pk::Model) so downstream consumers can run their own external CTC decoding stack (e.g., pyctcdecode + KenLM + hotwords) on top of parakeet.cpp’s already-computed CTC log-probabilities.

Changes:

Added pk::Model::{transcribe_pcm_ctc_logits, transcribe_16k_ctc_logits} to run mel + encoder + CTC head only and return the log-softmaxed [T, vocab+1] matrix.
Added parakeet_capi_transcribe_pcm_logits + parakeet_capi_free_logits, and bumped the C-API ABI to v6.
Added a new model-dependent test (test_capi_ctc_logits) and registered it in tests/CMakeLists.txt.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`src/model.hpp`	Declares new `Model` entrypoints to return CTC log-prob matrices.
`src/model.cpp`	Implements the mel+encoder path mirrored from `transcribe_16k`, then runs only the CTC head.
`include/parakeet_capi.h`	Documents ABI v6 and declares the new logits C-API functions and ownership contract.
`src/parakeet_capi.cpp`	Implements the new C-API function, allocates logits with `malloc`, and adds `free_logits`.
`tests/test_capi_ctc_logits.cpp`	Adds a new test intended to cover logits exposure behavior and error handling.
`tests/CMakeLists.txt`	Registers `test_capi_ctc_logits` and labels it as `model` with project-root working directory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    if (!out_logits || !out_T || !out_vocab_plus_1) return 1;
+    *out_logits = nullptr;
+    if (!ctx) return 1;
+    if (!ctx->model) { ctx->last_error = "context has no loaded model"; return 1; }
+    if (!samples || n_samples < 0) { ctx->last_error = "invalid samples buffer"; return 1; }


+        const std::string reference = model->transcribe_pcm(audio.samples, 16000, pk::Decoder::kCTC);
+
+        std::vector<float> logits;
+        int T = 0, vocab_plus_1 = 0;
+        model->transcribe_pcm_ctc_logits(audio.samples, 16000, logits, T, vocab_plus_1);
+


Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

tests/test_capi_ctc_logits.cpp:106

If you apply the earlier change to cache blank_id and tokenizer_pieces() (and potentially model.reset() to reduce peak RAM), this decode/reconstruction block should use those cached values rather than dereferencing model here. Otherwise you'll either keep the large model resident unnecessarily or run into redeclaration/lifetime issues after the refactor.

        const int blank_id = (int)model->config().blank_id;
        std::vector<int32_t> ids = pk::ctc_greedy(logits, T, vocab_plus_1, blank_id);
        const std::string reconstructed = pk::detokenize(
            model->loader().tokenizer_pieces(),
            pk::strip_special_tokens(model->loader().tokenizer_pieces(), ids));

tests/test_capi_ctc_logits.cpp:71

This test keeps a fully-loaded pk::Model alive while also loading a second full model via parakeet_capi_load(), which can significantly increase peak RAM (especially for 1.1B checkpoints) and make the test OOM. Capture the small metadata you need (blank_id + tokenizer pieces) and release the pk::Model before loading the C-API context to keep only one model resident at a time.

This issue also appears on line 102 of the same file.

        const std::string reference = model->transcribe_pcm(audio.samples, 16000, pk::Decoder::kCTC);

        parakeet_ctx* ctx = parakeet_capi_load(ctc_gguf);

src/model.cpp:202

The comment here claims prompt conditioning is irrelevant because a "CTC-only model always" isn't prompt-conditioned, but this method can be called on prompt-conditioned models too (and it does apply maybe_apply_prompt). Consider aligning this comment with transcribe_16k's wording to avoid misleading future readers.

    // 2b. Prompt conditioning (multilingual nemotron). No-op for non-prompt
    //     models — the case a CTC-only model always is in practice.

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

src/parakeet_capi.cpp:239

parakeet_capi_transcribe_pcm_logits treats a zero-sized logits matrix as an OOM error on allocators where malloc(0) returns NULL (allowed by the C standard). That makes the API behavior non-portable for edge cases like empty/too-short audio where T can legitimately be 0. Consider skipping the allocation/copy when logits.empty() and returning success with *out_logits == NULL (free_logits is already NULL-safe).

        float* buf = static_cast<float*>(std::malloc(logits.size() * sizeof(float)));
        if (!buf) { ctx->last_error = "out of memory"; return 1; }
        std::memcpy(buf, logits.data(), logits.size() * sizeof(float));

include/parakeet_capi.h:192

The new logits entry point doesn’t offer a target_lang / _lang variant, unlike the other offline/streaming C-API calls. Since Model::transcribe_pcm_ctc_logits supports target_lang, prompt-conditioned (nemotron) models currently can’t select a language prompt when requesting logits via the C-API. Consider adding parakeet_capi_transcribe_pcm_logits_lang(..., const char target_lang, ...) and delegating the non-lang function to it for consistency.

int parakeet_capi_transcribe_pcm_logits(parakeet_ctx* ctx, const float* samples,
                                        int n_samples, int sample_rate,
                                        float** out_logits, int* out_T,
                                        int* out_vocab_plus_1);

* feat(tdt): add opt-in N-best beam decoding Implement NeMo-compatible default TDT beam search, expose ranked hypotheses through Model, C API, and CLI surfaces, and add CPU/CUDA plus closed-loop NeMo parity coverage. Assisted-by: Codex:gpt-5 [Codex] * fix(tdt): harden N-best edge cases Validate requests before inference, reject malformed duration tables and non-progressing zero-duration expansions, and broaden real-model API coverage. [skip ci] Assisted-by: Codex:gpt-5 [Codex] * fix(tdt): score legal blank duration fallback Select the highest-scoring positive duration when zero is the only top-k blank duration, and cover the beam-1 fallback independently of a model. Assisted-by: Codex:gpt-5 [Codex] * fix(test): include invalid_argument definition Keep the TDT beam test portable across compilers instead of relying on a transitive standard-library include. Assisted-by: Codex:gpt-5 [Codex] --------- Co-authored-by: inlanger <[email protected]>

The C-API only ever returned decoded text/timestamps/JSON, even though CTCDecoder::forward already computes a log-softmaxed [T, vocab+1] matrix internally. External decoder stacks that do their own LM fusion (e.g. pyctcdecode + KenLM + hotwords) need that matrix directly instead of this library's own greedy/beam decode. Adds Model::transcribe_pcm_ctc_logits (mirrors transcribe_pcm through mel + encoder, including the long-audio tiling path, then runs the CTC head only, skipping decode) and the matching C-API entry point + parakeet_capi_free_logits, following the existing validate/try-catch/ last_error conventions used by every other capi_* function. Bumps the ABI version to 6. Verified against the real parakeet-ctc-1.1b checkpoint (tests/test_capi_ctc_logits.cpp, gated on PARAKEET_TEST_GGUF_CTC): argmax-greedy decode over the exposed matrix reproduces transcribe_pcm(..., kCTC)'s own greedy text byte-for-byte, and matches the fixture's known NeMo reference.

Verifies the C-API boundary contract directly: a model with no CTC head (a pure RNNT/TDT streaming model) makes parakeet_capi_transcribe_pcm_logits fail cleanly (nonzero return, *out_logits left NULL, last_error set) rather than letting ctc_head_tensor's std::runtime_error cross the C boundary. Gated on PARAKEET_TEST_GGUF_NO_CTC, independent of the existing self-consistency block.

- parakeet_capi_transcribe_pcm_logits: check ctx before the output pointers (so an invalid out-param can still set ctx->last_error), and zero *out_T/*out_vocab_plus_1 alongside *out_logits up front so every failure path leaves defined state, matching the header's documented contract. - tests/test_capi_ctc_logits.cpp block 1: call the actual parakeet_capi_load / parakeet_capi_transcribe_pcm_logits / parakeet_capi_free_logits C-API instead of Model::transcribe_pcm_ctc_logits directly, so the test exercises the C boundary and malloc/free contract on the success path too (pk::Model is now used only for the reference text and tokenizer/blank_id).

- tests/test_capi_ctc_logits.cpp block 1: cache blank_id/tokenizer_pieces and release the pk::Model (model.reset()) before loading a second full model via parakeet_capi_load — the two were resident simultaneously, nearly doubling peak RAM for a 1.1B checkpoint and risking CI OOM. - src/model.cpp: transcribe_16k_ctc_logits's prompt-conditioning comment claimed CTC-only models are never prompt-conditioned "in practice", which overclaims a current-catalog observation as if it were a property the function depends on. Reworded to match transcribe_16k's existing, accurate wording (no-op when prompt.present == false, full stop).

aanchan requested a review from Copilot July 28, 2026 05:02

Copilot started reviewing on behalf of aanchan July 28, 2026 05:02

View session

Copilot AI reviewed Jul 28, 2026

View reviewed changes

aanchan requested a review from Copilot July 28, 2026 05:47

Copilot started reviewing on behalf of aanchan July 28, 2026 05:48

View session

Copilot AI reviewed Jul 28, 2026

View reviewed changes

aanchan requested a review from Copilot July 28, 2026 06:02

Copilot started reviewing on behalf of aanchan July 28, 2026 06:02

View session

aanchan mentioned this pull request Jul 28, 2026

Add parakeet_capi_transcribe_pcm_logits: expose CTC log-probs mudler/parakeet.cpp#57

Merged

Copilot AI reviewed Jul 28, 2026

View reviewed changes

inlanger and others added 5 commits July 28, 2026 09:05

aanchan force-pushed the ctc-logits-capi branch from 7670078 to b73b426 Compare July 28, 2026 19:20

aanchan merged commit 043a4c5 into master Jul 28, 2026

Navigation Menu

Repository navigation

Add parakeet_capi_transcribe_pcm_logits: expose CTC log-probs - #1

Add parakeet_capi_transcribe_pcm_logits: expose CTC log-probs#1
aanchan merged 5 commits into
masterfrom
ctc-logits-capi

aanchan commented Jul 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

aanchan commented Jul 28, 2026

Summary

What's included

Next step

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants