Skip to content

fix(voice): encode streamed float32 audio as PCM16#3916

Merged
seratch merged 1 commit into
openai:mainfrom
FayezBast:fix/float32-streamed-STT-audio
Jul 23, 2026
Merged

fix(voice): encode streamed float32 audio as PCM16#3916
seratch merged 1 commit into
openai:mainfrom
FayezBast:fix/float32-streamed-STT-audio

Conversation

@FayezBast

Copy link
Copy Markdown
Contributor

Summary

OpenAISTTTranscriptionSession._stream_audio base64-encoded the raw audio
buffer and sent it in input_audio_buffer.append. StreamedAudioInput
explicitly accepts np.int16 | np.float32, but the session configures the
realtime transcription input as int16 PCM ({"type": "audio/pcm", "rate": 24000}). So for float32 input we shipped raw 32-bit IEEE-754 float bytes
(4 bytes/sample) to an endpoint expecting PCM16 (2 bytes/sample), producing
garbage/failed transcriptions. The batch/tracing path (_audio_to_base64)
already converted float32 → int16 correctly, so the streaming and tracing
paths disagreed.

This extracts the conversion into a shared _audio_buffer_to_base64 helper
(clip to [-1.0, 1.0], scale by 32767, cast to int16 — matching the
existing convention in voice/input.py) and uses it in both _stream_audio
and _audio_to_base64. The conversion is non-mutating (np.clip returns a
new array), so the buffer retained in _turn_audio_buffer for tracing is
unchanged and int16 input is unaffected.

Test plan

  • Parametrized coverage verifies:
    • int16 pass-through,
    • float32 clipping and PCM16 conversion,
    • exact transmitted bytes, and
    • input-buffer immutability.
  • code-change-verification: format, lint, typecheck, and full tests passed.

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh
  • I've confirmed all verification steps pass
  • If using Codex, I've run /review before submitting this PR

@seratch seratch added this to the 0.19.x milestone Jul 23, 2026
@seratch
seratch merged commit 0530398 into openai:main Jul 23, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants