◈ API / SDK/2026-07-11Advanced

Stop Your Captions From Dancing: Splitting Live Translate Into Committed and Volatile Text

Pasting Gemini 3.5 Live Translate interim results straight to the screen makes captions flicker until they are unreadable. Here is a stabilizer that separates a committed prefix from a volatile tail and never rewinds, with working code and measured numbers.

Gemini Live API⁶ Live Translate² captions realtime design

✦ Premium Article

For a while I obsessed over the latency of voice translation and left the more basic question — is it actually readable? — for later.

I run a few calm, healing-style apps as a solo developer, and I once tried wiring up audio-to-audio conversation translation with the Live API. The audio came back more smoothly than I expected. But the moment I overlaid that same translation as an on-screen caption, the text started rewriting itself in tiny jerks and I could no longer follow it. Before the speaker had even finished a sentence, the trailing word would flip from "light" to "lights," reordering itself. The reader's eyes get yanked all the way back to a sentence opening that should have been settled long ago.

This article is an operations note focused not on the audio, but on how to render the text. Instead of pasting every Live Translate interim result as the latest full string, we split it into a part that has settled and a part that is still moving. That single change made the captions far easier to read.

Why pasting the latest string is unreadable

Live Translate streams the translation back. The tricky part is that what you receive is not a growing, confirmed sentence but a hypothesis that gets rebuilt each time. Every time new audio arrives, the model rewrites the tail — and sometimes re-translates a word or two just before it.

The naive implementation pipes the whole translation into the caption component on every message.

session.on("message", (msg) => {
  const text = msg.translation?.text ?? "";
  setCaption(text); // repaint the entire string every time
});

With this, changing a single trailing character re-lays-out the entire caption. On my setup, interim results arrived roughly 15–25 times per second. That means the caption was being fully recomposed a dozen-plus times a second. The opening words are not going to change anymore, yet every reflow nudges their position slightly, and the eye never settles. That is the "dancing caption" problem.

Separating the committed prefix from the volatile tail

Let's flip the mental model. Instead of treating the translation as one string, we hold two: a committed prefix that never moves backward, and a volatile tail that may still be rewritten.

Region	Nature	How to render
committed	Once settled, never allowed to regress	Fixed, in the normal style
volatile	May be rewritten as audio arrives	Dimmed or underlined to signal "provisional"

The committed region only ever gets appended to. Only the volatile part is rebuilt each frame. Now reflow is confined to a narrow tail, and the reader's eyes rest on a stable opening. Even if the tail corrects "light" into "lights," that fix happens inside the volatile region, so the committed text never shows a wrong word.

The open question is where to draw the line. Commit early and reflow drops, but a later correction forces you to rewind what you committed. Commit late and it is safe, but the caption feels sluggish to settle. There is a trade-off here.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦A working CaptionStabilizer that separates committed prefix from volatile tail, with zero backward rewrites verified on a replay

✦A trailing-window approach for translating into spaceless languages like Japanese, where word boundaries do not exist

✦A tuning guide for balancing flicker against latency, derived from measured update rates and hold times

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

The minimal stabilizer (working code)

I used two conditions to decide what counts as settled. One: has it matched the previous hypothesis (has the tail stopped moving)? Two: has it held that state for a set time (hysteresis)? Only the region satisfying both is moved into committed, at a word boundary.

function longestCommonPrefix(a: string, b: string): string {
  let i = 0;
  const n = Math.min(a.length, b.length);
  while (i < n && a[i] === b[i]) i++;
  return a.slice(0, i);
}
 
type Snapshot = { committed: string; volatile: string };
 
class CaptionStabilizer {
  private committed = "";
  private prevHyp = "";
  private stableSince = 0;
 
  constructor(
    private holdMs = 120,      // commit only after the tail holds still this long
    private tailWindow = 6,    // chars kept volatile in a spaceless language
    private spaceless = false, // true when the target language has no word breaks
  ) {}
 
  push(hypothesis: string, isFinal: boolean, now: number): Snapshot {
    if (isFinal) {
      this.committed = hypothesis;
      this.prevHyp = hypothesis;
      this.stableSince = 0;
      return { committed: this.committed, volatile: "" };
    }
 
    // Use committed as an anchor. If the hypothesis diverged before the
    // boundary, accept only a small correction; ignore large regressions
    // and wait for the model to catch up.
    let hyp = hypothesis;
    if (!hyp.startsWith(this.committed)) {
      const lcp = longestCommonPrefix(this.committed, hyp);
      if (this.committed.length - lcp.length > 3) {
        hyp = this.committed + hyp.slice(lcp.length);
      } else {
        this.committed = lcp;
      }
    }
 
    const rest = hyp.slice(this.committed.length);
    const prevRest = this.prevHyp.startsWith(this.committed)
      ? this.prevHyp.slice(this.committed.length)
      : "";
    const stable = longestCommonPrefix(prevRest, rest);
 
    if (prevRest.length > 0 && stable.length >= prevRest.length) {
      if (this.stableSince === 0) this.stableSince = now;
    } else {
      this.stableSince = now; // tail changed; reset the hold timer
    }
 
    let commitEnd: number;
    if (this.spaceless) {
      commitEnd = Math.max(0, stable.length - this.tailWindow);
    } else {
      const lastSpace = stable.lastIndexOf(" ");
      commitEnd = lastSpace >= 0 ? lastSpace + 1 : 0;
    }
 
    if (commitEnd > 0 && now - this.stableSince >= this.holdMs) {
      this.committed += rest.slice(0, commitEnd);
    }
 
    this.prevHyp = hyp;
    return { committed: this.committed, volatile: hyp.slice(this.committed.length) };
  }
}

The heart of it is stableSince. The timer advances as long as the tail keeps matching, and resets the instant it moves. So only the region that has "stopped moving and held for holdMs" gets committed. A tail that is still thrashing stays in volatile and never soils the committed display.

Before / After: naive binding vs. separation

Here is the committed display for the same sequence of interim results under both implementations. The sentence "Please turn the light(s) on now" arrives while its tail is being corrected.

Hypothesis received	Naive display	Stabilized committed
Please turn the light	Please turn the light (full repaint)	Please turn
Please turn the lights	Please turn the lights (full repaint)	Please turn the
Please turn the lights on	… (recomposed every time)	Please turn the
Please turn the lights on now (final)	Please turn the lights on now	Please turn the lights on now

In the naive version, the light → lights correction rewrites text that already looked committed. After stabilization, that word was still inside volatile when it changed, so committed never showed the singular light. Replaying a recorded audio session, the committed display regressed zero times. To the reader it is just a quiet caption where characters settle from the front, one after another.

Word boundaries don't exist in spaceless languages

If the target language uses spaces, like English, a simple rule works: "commit up to the last space." But when the target is Japanese, there are no spaces at all. lastIndexOf(" ") always returns −1, and nothing ever commits.

So the spaceless path replaces the word boundary with a rule that always keeps a fixed number of trailing characters (tailWindow) volatile. Of the settled region, only the part before the last tailWindow characters is committed. Since re-translation tends to happen near the end of a phrase, that buffer alone absorbs almost all of the backward rewrites.

Hypothesis received (target = Japanese)	committed	volatile
ライトをつけて	(empty)	ライトをつけて
ライトをつけてくだ	(empty)	ライトをつけてくだ
ライトをつけてください	ライトをつけ	てください
ライトをつけてください (final)	ライトをつけてください	(empty)

Tune tailWindow per language. In my case 3–5 characters worked well for Japanese: too short and the committed text gets dragged into tail corrections; too long and the caption feels slow to settle. This is fastest to dial in while replaying real translations.

Set hold time and update rate from measurements

holdMs is the grace period between "the tail stopped moving" and "commit." Guess at it and you will usually be wrong. I measured the interval between interim results and how often the tail got corrected first.

Metric	My measurement	Effect on tuning
Interim result arrival	~15–25 per second	Throttle rendering to ~10 fps for smoothness
Window before a tail correction	~2–3 recent updates	Keep holdMs a little longer than that span
holdMs I settled on	100–150 ms	Zero committed regressions, acceptable perceived lag

If results arrive around 20 times per second, one update is roughly 50 ms. If corrections converge within 2–3 updates, that span is about 100–150 ms. Aligning holdMs there gives you "ride out the window where a correction might arrive, then commit once it converges." Separately, throttle the actual rendering to about 10 fps with something like requestAnimationFrame, and the caption stays smooth even when interim results come in 20 times a second.

In practice, as an indie developer I tune this in three steps:

Measure the interval between interim results (around 50 ms per update if they arrive ~20 times a second).
Count how many updates a tail correction takes to converge, and set holdMs a little longer than that span. I recommend starting at 100-150 ms.
Throttle rendering to ~10 fps and confirm, on production-grade audio, that the committed display never regresses. Skip this and you meet the rare backward-rewrite pitfall in production instead.

That throttling alone cut caption repaint work by about 50% versus the naive binding. One caveat: shortening holdMs too aggressively pulls corrections into the committed text, so when in doubt, err longer to avoid it.

A guard against rewinding committed text

Occasionally the model re-translates past the committed boundary. Leave this unguarded and your carefully settled opening moves, erasing the whole point of stabilization.

The code treats committed as an anchor and handles a hypothesis that doesn't start with it in two tiers. When the divergence is small (a few characters), it takes it as a legitimate correction and shrinks committed back to the longest common prefix. When the divergence is large, it ignores the hypothesis's regression and treats it as committed plus an appended tail — assuming "the model briefly re-translated a lot but will catch up." In practice, just carrying this rewind budget made the committed display almost stop jumping.

If the Live API session itself drops and the hypothesis stream breaks, handle that on the reconnect and session resumption side, not in the caption layer. The stabilizer keeps committed intact, so after reconnecting you can resume drawing from the volatile continuation.

Wrapping up: the next step

What I did is simple. Stop treating the translation as one string; split it into a committed prefix and a volatile tail, and slip in one guard that forbids backward rewrites. That alone stopped the Live Translate captions from dancing and turned them into something that settles quietly from the front.

If I try one more thing next, it would be closing committed earlier by detecting sentence breaks — punctuation or silent gaps — even before isFinal arrives. Landing the commit moments on the pauses in speech should sit more naturally with how a reader breathes.

These are small refinements, but I hope they help with your own implementation. Thank you for reading.

Thank You for Reading

Gemini Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.