Making TTS seams inaudible: prosody continuity and silence boundaries for long-form narration

Killing the click (zero-crossing snap and edge fade)

Even with the intonation carried across, a small click can remain at the waveform joint. The cause is that the audio edge is cut at a point where the sample does not cross zero. When a nonzero amplitude switches to the next silence (zero), a step forms and it snaps.

The fix has two parts. First trim the edge to a point near zero, then apply a very short fade. Around 8 milliseconds is enough -- it flattens the step without eating into the phonemes.

import numpy as np
 
SR = 24000        # Gemini TTS returns 24kHz 16bit mono
FADE = 192        # 8ms @ 24kHz
 
def pcm_to_np(pcm: bytes) -> np.ndarray:
    return np.frombuffer(pcm, dtype=np.int16).astype(np.float32)
 
def np_to_pcm(a: np.ndarray) -> bytes:
    return np.clip(a, -32768, 32767).astype(np.int16).tobytes()
 
def snap_zero_crossing(a: np.ndarray, head: bool, window: int = 240) -> np.ndarray:
    n = min(window, len(a))
    if n == 0:
        return a
    if head:
        idx = int(np.argmin(np.abs(a[:n])))
        return a[idx:]
    idx = int(np.argmin(np.abs(a[-n:])))
    return a[: len(a) - (n - 1 - idx)]
 
def edge_fade(a: np.ndarray, n: int = FADE) -> np.ndarray:
    n = min(n, len(a))
    if n == 0:
        return a
    ramp = np.linspace(0.0, 1.0, n, dtype=np.float32)
    a = a.copy()
    a[:n] *= ramp
    a[-n:] *= ramp[::-1]
    return a

With naive concatenation, the click was audible at roughly a third of the chunk boundaries. After the zero-crossing snap and edge fade, it sits low enough that you have to hunt for it. Nothing flashy -- but the quieter the source, the more it matters.

Evening out the pauses with punctuation (adaptive silence)

A fixed pause length will feel wrong somewhere. Make the gap after a sentence equal to the gap after a paragraph and the paragraphs read as one continuous run. Give no pause after a closing quotation mark and the speaker seems to barrel into the next line without a breath.

So we vary the silence by the sentence-ending punctuation. Here are the values that settled well for me. The best figures shift by material, so use these as a starting point.

def gap_for(sentence: str, is_paragraph_end: bool) -> float:
    if is_paragraph_end:
        return 0.60
    s = sentence.rstrip()
    if s.endswith(("”", "\"", "」", "』")):
        return 0.36
    if s.endswith(("!", "?")):
        return 0.34
    return 0.28
 
def silence(sec: float) -> np.ndarray:
    return np.zeros(int(SR * sec), dtype=np.float32)

Re-rendering one chunk (an idempotent seam ledger)

Rebuilding the whole track just to fix one sentence wastes both time and money. Worse, priming is non-deterministic: every rebuild nudges the intonation across the entire piece, so parts you never touched drift anyway.

The answer is to record, per segment, what each clip was generated from, and re-render only what changed.

{
  "voice": "Kore",
  "sample_rate": 24000,
  "segments": [
    {"idx": 0, "text_hash": "a1b2c3d4e5f6", "style_hash": "9f2c7a", "gap_after": 0.28, "audio": "seg_000.pcm"},
    {"idx": 1, "text_hash": "0b1c2d3e4f5a", "style_hash": "9f2c7a", "gap_after": 0.60, "audio": "seg_001.pcm"}
  ]
}

import hashlib, json, os
 
def h(s: str) -> str:
    return hashlib.sha1(s.encode("utf-8")).hexdigest()[:12]
 
def render(sentences, gaps, ledger_path, seg_dir):
    os.makedirs(seg_dir, exist_ok=True)
    style_hash = h(STYLE + VOICE)
    old = json.load(open(ledger_path)) if os.path.exists(ledger_path) else {"segments": []}
    prev = {seg["idx"]: seg for seg in old.get("segments", [])}
    segments, prev_tail = [], ""
    for i, s in enumerate(sentences):
        th = h(s)
        cached = prev.get(i)
        reuse = (
            cached
            and cached["text_hash"] == th
            and cached.get("style_hash") == style_hash
            and os.path.exists(os.path.join(seg_dir, cached["audio"]))
        )
        name = cached["audio"] if reuse else f"seg_{i:03d}.pcm"
        if not reuse:
            open(os.path.join(seg_dir, name), "wb").write(synth(s, prev_tail))
        segments.append({"idx": i, "text_hash": th, "style_hash": style_hash,
                         "gap_after": gaps[i], "audio": name})
        prev_tail = s[-24:]
    json.dump({"voice": VOICE, "sample_rate": SR, "segments": segments},
              open(ledger_path, "w"), ensure_ascii=False, indent=2)
    return segments
 
def assemble(segments, seg_dir) -> bytes:
    out = []
    for seg in segments:
        pcm = open(os.path.join(seg_dir, seg["audio"]), "rb").read()
        a = pcm_to_np(pcm)
        a = snap_zero_crossing(a, head=True)
        a = snap_zero_crossing(a, head=False)
        a = edge_fade(a)
        out.append(a)
        out.append(silence(seg["gap_after"]))
    return np_to_pcm(np.concatenate(out))

Keeping generation and assembly separate means you never re-synthesize just to tweak the polishing parameters. Silence lengths and fades can be rebuilt from the PCM on disk as many times as you like.

Field notes the docs do not mention

A few things I stumbled on that the documentation leaves out.

Do not mistake the sample rate. Gemini's TTS is 24kHz. Write 16kHz or 44.1kHz into a WAV header and playback runs fast or drags. It is easy to blame the voice quality instead.

Always check finish_reason. Anything other than STOP means the audio was cut short and stitched anyway. Before you suspect the seam, confirm the chunk came back whole.

Decide how far a priming change propagates. Edit one sentence and the next sentence's primer changes too. Strictly you should re-render the following segment as well, but tolerating a faint intonation difference and confining the change to the edited chunk is a practical policy. My rule of thumb: re-render the next one for an in-paragraph edit, stop at a paragraph head.

Drop priming at paragraph heads. Where the topic shifts hard, you sometimes want the intonation to reset. Pass an empty prev_tail for the first sentence of a paragraph and let it read as a new flow.

Key the cache on the input. Since priming is non-deterministic, the audio file itself is where you guarantee reproducing the same sound. As long as the text and style hashes match, treat the PCM on disk as the source of truth. That is what makes re-rendering safe.

In closing -- the first move

None of this is flashy. Priming carries the intonation, the zero-crossing snap and edge fade remove the click, punctuation varies the silence, and the ledger localizes re-rendering. Four layers, all of them small acts of polishing stacked up.

Start by recording one article both ways -- naively joined and built with this design -- and listen to them back to back. You will find one boundary where your ear catches. Begin by fixing that single spot; that is enough.

Erasing the small snags that linger in the ear is quiet, unglamorous work, but it steadies the experience of the person listening. I hope it helps in your own build.