fix(voice): encode streamed float32 audio as PCM16#3916
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OpenAISTTTranscriptionSession._stream_audiobase64-encoded the raw audiobuffer and sent it in
input_audio_buffer.append.StreamedAudioInputexplicitly accepts
np.int16 | np.float32, but the session configures therealtime transcription input as int16 PCM (
{"type": "audio/pcm", "rate": 24000}). So for float32 input we shipped raw 32-bit IEEE-754 float bytes(4 bytes/sample) to an endpoint expecting PCM16 (2 bytes/sample), producing
garbage/failed transcriptions. The batch/tracing path (
_audio_to_base64)already converted float32 → int16 correctly, so the streaming and tracing
paths disagreed.
This extracts the conversion into a shared
_audio_buffer_to_base64helper(clip to
[-1.0, 1.0], scale by32767, cast toint16— matching theexisting convention in
voice/input.py) and uses it in both_stream_audioand
_audio_to_base64. The conversion is non-mutating (np.clipreturns anew array), so the buffer retained in
_turn_audio_bufferfor tracing isunchanged and int16 input is unaffected.
Test plan
code-change-verification: format, lint, typecheck, and full tests passed.Checks
.agents/skills/code-change-verification/scripts/run.sh/reviewbefore submitting this PR