Gemini-3.8-live hallucinates user speech (“¿Qué?”) when nobody has said anything (3.1 does not)

Setup:

Model gemini-3.8-live, API v1alpha, audio in / audio out
Input: 16 kHz, 16-bit mono PCM, sent in 50 ms chunks (LiveKit Agents 1.8.2 Google plugin; reproduced on a raw WebSocket too)
automaticActivityDetection: startOfSpeechSensitivity = START_SENSITIVITY_HIGH, silenceDurationMs = 600
Input transcription enabled; in-car assistant, users speak English / French

Symptom:

While the session is listening and nobody says anything, the model still reports user speech. The input transcription shows “¿Qué?” and the model answers it, as if the user had asked something. Less often the phantom is another single word (“C’est”, “A”, “so”).

The phantom word is almost always Spanish, even in English or French conversations. Our prompt says to follow the user’s spoken language, so after a phantom “¿Qué?” the whole conversation switches to Spanish.

Minimal reproduction:

Clip: 2 s of silence, then 450 ms of background sound with no speech in it (peak ≈ −35 dBFS), cut from the audio our agent actually sent.

gemini-3.8-live: “¿Qué?” 5/5, at every gain from −6 dB to +12 dB.
gemini-3.1-flash-live-preview, same clip and same config: 0/5 at every gain.
Ruled out on our side: DTX, digital silence, network gaps, AGC.

What we tried:

START_SENSITIVITY_LOW: no more phantoms (0/9), but it also drops real short words (“Hey”, “Oui”, ~0.5 s) 2/2, so we cannot use it.
A client-side VAD gate before sending audio works locally, but we’d rather not second-guess the model’s own VAD.

Questions:
1.Has anyone else run into this with gemini-3.8-live, phantom “¿Qué?” or other one-word transcripts when the user said nothing?
2. Is there a recommended way to suppress this without lowering start sensitivity?

Update: reproduced with the official SDK only, no prompt, no tools (repro attached)

Attached: que_repro_gemini38.zip. It contains que_isolation.py, the two clips, a README and our results.

que_isolation.py uses only the google-genai SDK on the Gemini Developer API (API key, not Vertex): no system instruction, no tools, no third-party audio stack. It streams each clip in real-time 100 ms chunks, followed by 3 s of silence, and counts a run as a phantom when input_transcription is non-empty. None of the clips contain speech.

Results, 5 runs per cell:

Model Clip VAD default VAD START_SENSITIVITY_HIGH
gemini-3.8-live que_trigger.wav 0/5 5/5 "¿Qué?"
gemini-3.8-live que_trigger_boost20dB.wav 0/5 5/5 "¿Qué?"
gemini-3.8-live digital silence 0/5 0/5
gemini-3.1-flash-live-preview que_trigger.wav 0/5 0/5
gemini-3.1-flash-live-preview que_trigger_boost20dB.wav 0/5 0/5
gemini-3.1-flash-live-preview digital silence 0/5 0/5

Leaving startOfSpeechSensitivity unset showed no phantoms. It behaves differently from an explicit START_SENSITIVITY_HIGH.

Correction to the numbers in the original post: our earlier raw-WebSocket test was 3/3 on 3.8 and 0/3 on 3.1, at gains from −6 to +12 dB.
The 5/5 figures are from the SDK run above, at 0 and +20 dB.

que_repro_gemini38.zip

@r09528039

Hi @r09528039, thanks for all the details! Could you share the exact code snippet of your receive loop? We'd love to try reproducing this on our end using the exact logic you have set up.
Thanks

@Venkaiahbabuneelam

Hello @Venkaiahbabuneelam,
Sure! Please refer to the attachment (que_repro_gemini38.zip) from my previous comment, thanks.

@Venkaiahbabuneelam

que_repro_gemini38_v3.zip

Hi @Venkaiahbabuneelam, here is the exact code inline. It is the same logic as que_isolation.py in the attached que_repro_gemini38_v3.zip, which also contains the audio files listed below.

import asyncio, os, wave
from google import genai
from google.genai import types

# Gemini Developer API key (not Vertex)
client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"],
                      http_options={"api_version": "v1alpha"})

CONFIG = types.LiveConnectConfig(
    response_modalities=["AUDIO"],
    input_audio_transcription=types.AudioTranscriptionConfig(),
    output_audio_transcription=types.AudioTranscriptionConfig(),
    realtime_input_config=types.RealtimeInputConfig(
        automatic_activity_detection=types.AutomaticActivityDetection(
            start_of_speech_sensitivity=types.StartSensitivity.START_SENSITIVITY_HIGH)),
    # no system_instruction, no tools
)

async def run_once(path="que_trigger.wav"):
    with wave.open(path) as w:
        pcm = w.readframes(w.getnframes())
    pcm += bytes(16000 * 2 * 3)                      # + 3 s of silence
    heard, said = [], []
    async with client.aio.live.connect(model="gemini-3.8-live", config=CONFIG) as s:

        async def send():                            # real-time paced, 100 ms chunks
            for i in range(0, len(pcm), 3200):
                await s.send_realtime_input(
                    audio=types.Blob(data=pcm[i:i + 3200], mime_type="audio/pcm;rate=16000"))
                await asyncio.sleep(0.1)

        async def recv():                            # the receive loop
            while True:                              # receive() ends at each turn_complete
                async for m in s.receive():
                    sc = m.server_content
                    if sc and sc.input_transcription and sc.input_transcription.text:
                        heard.append(sc.input_transcription.text)
                    if sc and sc.output_transcription and sc.output_transcription.text:
                        said.append(sc.output_transcription.text)

        rx = asyncio.create_task(recv())
        await send()
        await asyncio.sleep(2)
        rx.cancel()
    return "".join(heard), "".join(said)

for k in range(5):
    print(asyncio.run(run_once()))

Expected: heard stays empty, because none of the clips contain speech.

Actual: on gemini-3.8-live + START_SENSITIVITY_HIGH, every run returns a phantom user transcript, and the model replies to it (in Spanish), e.g. ('¿Qué?', 'Hola, ¿en qué te puedo ayudar?'). Re-checked today (2026-10-27) with the exact code above.

The zip contains three independent events from our in-car assistant. Each is 16 kHz, mono, 16-bit, and none contains speech: Silero VAD's max speech probability is 0.07–0.39 on every clip, versus 1.00 for a real spoken "Oui".

File Source 3.8 + HIGH 3.1 + HIGH
que_trigger.wav audio our agent streamed to the Live API 5/5 "¿Qué?" 0/4
car_input_2.wav audio our agent streamed to the Live API, another event 5/5 "y" 0/4
car_mic_3.wav raw head-unit microphone recording, another event 5/5 "¿Qué?" 0/4
digital silence — 0/5 0/4

Also:

With START_SENSITIVITY_LOW, que_trigger.wav gave no phantom in 9/9 runs.
With start_of_speech_sensitivity not set, que_trigger.wav gave no phantom in 17/17 runs, although the docs say the default is START_SENSITIVITY_HIGH. On 3.8, "not set" behaves differently from an explicit HIGH.
Verified with google-genai 2.10.0 and 2.33.0, the latter in a clean python:3.11-slim container with only pip install google-genai.

@r09528039

Hi @r09528039,
Thank you for providing the minimal reproduction code and audio clips! We have reproduced and confirmed the issue on our end (3/3 runs on gemini-3.8-live with START_SENSITIVITY_HIGH producing a phantom '¿Qué?' input transcription and Spanish reply on que_trigger.wav, compared to 0/3 with default VAD and 0/3 on gemini-3.1-flash-live-preview). We truly appreciate you flagging this issue, and we will file a bug internally.
Thanks

Navigation Menu

Repository navigation

Gemini-3.8-live hallucinates user speech (“¿Qué?”) when nobody has said anything (3.1 does not) #3193

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Gemini-3.8-live hallucinates user speech (“¿Qué?”) when nobody has said anything (3.1 does not) #3193

Description

Activity

r09528039 commented on Oct 22, 2026

Venkaiahbabuneelam commented on Oct 22, 2026

r09528039 commented on Oct 22, 2026

r09528039 commented on Oct 27, 2026

Venkaiahbabuneelam commented on Oct 27, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions