Streaming Gemini TTS: concatenate the PCM, write the WAV header once

Alongside my work as an indie developer, I turn some of my articles into audio and publish them on stand.fm. Send the text, take the audio back, write one file. Nothing clever about it.

But listening to my own episodes, the gap between pressing play and hearing the first word kept bothering me. Of course it did — I was waiting for the whole synthesis to finish before writing anything.

streamGenerateContent now works with gemini-3.1-flash-tts-preview, so I sat down to see whether the first chunk could fill that gap. The latency turned out to be the easy part. The hard part was realizing what the chunks actually are.

Streamed audio is not an audio file

The response shape is the same as any other generation: an array under candidates[0].content.parts. For audio, inlineData sits where the text would be.

If you call REST directly, inlineData.data is a base64 string. If you use the Python SDK (google-genai), it has already been decoded into bytes. The type depends on your transport, so decide early which one your receiver expects.

The other field that matters is mimeType. On my runs it came back as:

audio/L16;codec=pcm;rate=24000

L16 means 16-bit linear PCM, and rate=24000 is the sample rate. In other words, you are getting raw waveform data with no header — not an MP3, not a WAV.

I read the rate out of mimeType rather than hardcoding it. If the value ever changes with a different model or voice config, a hardcoded number will only announce itself as audio that plays at the wrong speed.

import re
 
def parse_rate(mime_type: str, default: int = 24000) -> int:
    m = re.search(r"rate=(\d+)", mime_type or "")
    return int(m.group(1)) if m else default
 
# audio/L16;codec=pcm;rate=24000 -> 24000
# audio/pcm;rate=16000           -> 16000
# audio/pcm                      -> 24000 (fallback)

The receiver itself is small. For every chunk, if there is an inlineData blob, append its bytes.

import wave
from google import genai
from google.genai import types
 
client = genai.Client(api_key="YOUR_API_KEY")
MODEL = "gemini-3.1-flash-tts-preview"
 
def synthesize(text: str, out_path: str) -> None:
    pcm = bytearray()
    rate = None
 
    stream = client.models.generate_content_stream(
        model=MODEL,
        contents=text,
        config=types.GenerateContentConfig(
            response_modalities=["AUDIO"],
            speech_config=types.SpeechConfig(
                voice_config=types.VoiceConfig(
                    prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Kore")
                )
            ),
        ),
    )
 
    for chunk in stream:
        for part in (chunk.candidates[0].content.parts or []):
            blob = getattr(part, "inline_data", None)
            if blob is None or not blob.data:
                continue
            if rate is None:
                rate = parse_rate(blob.mime_type)
            pcm += blob.data          # append, nothing else
 
    with wave.open(out_path, "wb") as w:
        w.setnchannels(1)
        w.setsampwidth(2)             # 16-bit
        w.setframerate(rate or 24000)
        w.writeframes(bytes(pcm))     # the header is written exactly once

That last line — writing once, at the end — is the whole point of the next section.

Wrap every chunk in a header and you get one tenth of the audio

My first version saved each incoming chunk as its own WAV file and concatenated them afterwards. The result opens fine. It even starts playing. Then it stops.

To see exactly what was happening, I measured it with synthetic audio: ten chunks of 0.6 seconds each at 24 kHz, 16-bit mono, for a total of 6.0 seconds, written out two different ways.

The sizes are nearly identical. The difference is one 44-byte header versus ten, and not a single byte of waveform is missing. Yet only the first 0.6 seconds — one tenth of the audio — ever plays.

The data chunk of a WAV header declares how many bytes of audio follow it. The leading header says "28,800 bytes of audio come next," so the player stops there. The remaining nine chunks, headers and all, sit outside the declared region.

At 24 kHz, 16-bit, mono, one second of audio is:

24,000 samples/s × 2 bytes × 1 channel = 48,000 bytes/s

288,000 ÷ 48,000 = 6.0 seconds. Being able to derive duration from a byte count is a genuine convenience of raw PCM. However many chunks it arrived in, you concatenate and divide.

One more number worth keeping in mind: holding those 288,000 bytes as base64 costs 384,000 bytes, exactly 1.3333×. That is 96,000 extra bytes for six seconds of speech, so a ten-minute reading is not a rounding error. Decode to bytes as the chunks arrive rather than accumulating strings.

Chunk boundaries do not respect sample boundaries

A sample is two bytes. A chunk that arrives with an odd byte count is therefore cut in the middle of a sample.

As long as you keep everything as bytes and concatenate, nothing goes wrong: the trailing byte of one chunk meets the leading byte of the next exactly where it should.

The dangerous version is converting each chunk to numeric samples before adding it to your buffer. You end up discarding the odd byte at every boundary.

I measured that too, feeding five chunks of 7,201 / 7,203 / 7,199 / 7,205 / 7,192 bytes (36,000 bytes total) through both approaches.

Four bytes were lost. That was enough to corrupt 14,386 of the 17,998 samples. Once you are off by one byte, the high and low halves of every following sample swap places, and the waveform from that point on is something else entirely.

To the ear it sounds like static creeping in partway through. I nearly spent an afternoon rewriting prompts before accepting that the receiver was at fault, not the model.

Carrying the remainder forward removes the problem:

import array
 
class PcmDecoder:
    """Survives chunk boundaries that fall inside a sample."""
    def __init__(self) -> None:
        self._carry = b""
 
    def feed(self, chunk: bytes) -> array.array:
        buf = self._carry + chunk
        n = len(buf) // 2 * 2      # round down to a whole sample
        self._carry = buf[n:]      # keep the remainder for next time
        samples = array.array("h")
        samples.frombytes(buf[:n])
        return samples

Running the same five chunks through this version recovered all 18,000 samples, every one matching the reference, with 0 bytes left in the carry buffer. You need this whenever you drive a level meter, a silence detector, or any live analysis.

If you are only writing a file, you never need numeric samples at all. I keep both paths around and pick per use case.

Getting playback started sooner

Back to the original goal. If you are piping audio into a browser or a player, you can send the 44-byte header before any audio arrives.

import struct
 
def wav_header(rate: int, channels: int = 1, width: int = 2,
               data_size: int = 0x7FFFFFFF) -> bytes:
    byte_rate = rate * channels * width
    block_align = channels * width
    return (
        b"RIFF" + struct.pack("<I", 0xFFFFFFFF) + b"WAVE"
        + b"fmt " + struct.pack("<IHHIIHH", 16, 1, channels, rate,
                                byte_rate, block_align, width * 8)
        + b"data" + struct.pack("<I", data_size)
    )

Since the length is not known yet, the data size is a value large enough never to be reached. Writing one out and reading it back, the declared frame count came to 1,073,741,823 while the actual payload was a single second — 48,000 bytes.

Players that read forward handle this fine. The 48,000 bytes I read back were bit-for-bit identical to the PCM I fed in. The cost is that the seek bar has no idea how long the audio really is. For anything where total duration is part of the product, this trick is the wrong choice.

When you are writing a file, reserve the 44 bytes up front and seek back to fill in the sizes once you are done. Python's wave module does exactly that on close(), which is why the synthesize() function above never has to think about it.

When not to stream at all

Having written all of that, I should admit that my actual episodes do not use streaming.

An article can be narrated before it is published. By the time a listener presses play, the audio should already exist as a file. Streaming earns its keep only when the text is decided in the moment.

Streaming changes how the wait is perceived; it does not make synthesis faster. Generating at request time what you could have generated ahead of time charges your listeners twice — once in latency, once in API calls. I wrote up how I draw that line, including how to estimate the call count before you build, in moving app AI work from runtime calls to a pre-ship batch pass.

Wrapping up

Streaming Gemini TTS comes down to one sentence: you receive PCM fragments, and the header belongs to the whole, written once.

If your current implementation saves an audio file per chunk, measure the duration of what you produced. If it matches the first chunk and nothing more, you have found your bug. It took me half a day of blaming generation parameters before I looked in the right place.

Thanks for reading — I hope it saves you that afternoon.