Stop Your Captions From Dancing: Splitting Live Translate Into Committed and Volatile Text

The minimal stabilizer (working code)

I used two conditions to decide what counts as settled. One: has it matched the previous hypothesis (has the tail stopped moving)? Two: has it held that state for a set time (hysteresis)? Only the region satisfying both is moved into committed, at a word boundary.

function longestCommonPrefix(a: string, b: string): string {
  let i = 0;
  const n = Math.min(a.length, b.length);
  while (i < n && a[i] === b[i]) i++;
  return a.slice(0, i);
}
 
type Snapshot = { committed: string; volatile: string };
 
class CaptionStabilizer {
  private committed = "";
  private prevHyp = "";
  private stableSince = 0;
 
  constructor(
    private holdMs = 120,      // commit only after the tail holds still this long
    private tailWindow = 6,    // chars kept volatile in a spaceless language
    private spaceless = false, // true when the target language has no word breaks
  ) {}
 
  push(hypothesis: string, isFinal: boolean, now: number): Snapshot {
    if (isFinal) {
      this.committed = hypothesis;
      this.prevHyp = hypothesis;
      this.stableSince = 0;
      return { committed: this.committed, volatile: "" };
    }
 
    // Use committed as an anchor. If the hypothesis diverged before the
    // boundary, accept only a small correction; ignore large regressions
    // and wait for the model to catch up.
    let hyp = hypothesis;
    if (!hyp.startsWith(this.committed)) {
      const lcp = longestCommonPrefix(this.committed, hyp);
      if (this.committed.length - lcp.length > 3) {
        hyp = this.committed + hyp.slice(lcp.length);
      } else {
        this.committed = lcp;
      }
    }
 
    const rest = hyp.slice(this.committed.length);
    const prevRest = this.prevHyp.startsWith(this.committed)
      ? this.prevHyp.slice(this.committed.length)
      : "";
    const stable = longestCommonPrefix(prevRest, rest);
 
    if (prevRest.length > 0 && stable.length >= prevRest.length) {
      if (this.stableSince === 0) this.stableSince = now;
    } else {
      this.stableSince = now; // tail changed; reset the hold timer
    }
 
    let commitEnd: number;
    if (this.spaceless) {
      commitEnd = Math.max(0, stable.length - this.tailWindow);
    } else {
      const lastSpace = stable.lastIndexOf(" ");
      commitEnd = lastSpace >= 0 ? lastSpace + 1 : 0;
    }
 
    if (commitEnd > 0 && now - this.stableSince >= this.holdMs) {
      this.committed += rest.slice(0, commitEnd);
    }
 
    this.prevHyp = hyp;
    return { committed: this.committed, volatile: hyp.slice(this.committed.length) };
  }
}

The heart of it is stableSince. The timer advances as long as the tail keeps matching, and resets the instant it moves. So only the region that has "stopped moving and held for holdMs" gets committed. A tail that is still thrashing stays in volatile and never soils the committed display.

Before / After: naive binding vs. separation

Here is the committed display for the same sequence of interim results under both implementations. The sentence "Please turn the light(s) on now" arrives while its tail is being corrected.

In the naive version, the light → lights correction rewrites text that already looked committed. After stabilization, that word was still inside volatile when it changed, so committed never showed the singular light. Replaying a recorded audio session, the committed display regressed zero times. To the reader it is just a quiet caption where characters settle from the front, one after another.

Word boundaries don't exist in spaceless languages

If the target language uses spaces, like English, a simple rule works: "commit up to the last space." But when the target is Japanese, there are no spaces at all. lastIndexOf(" ") always returns −1, and nothing ever commits.

So the spaceless path replaces the word boundary with a rule that always keeps a fixed number of trailing characters (tailWindow) volatile. Of the settled region, only the part before the last tailWindow characters is committed. Since re-translation tends to happen near the end of a phrase, that buffer alone absorbs almost all of the backward rewrites.

Tune tailWindow per language. In my case 3–5 characters worked well for Japanese: too short and the committed text gets dragged into tail corrections; too long and the caption feels slow to settle. This is fastest to dial in while replaying real translations.

Set hold time and update rate from measurements

holdMs is the grace period between "the tail stopped moving" and "commit." Guess at it and you will usually be wrong. I measured the interval between interim results and how often the tail got corrected first.

If results arrive around 20 times per second, one update is roughly 50 ms. If corrections converge within 2–3 updates, that span is about 100–150 ms. Aligning holdMs there gives you "ride out the window where a correction might arrive, then commit once it converges." Separately, throttle the actual rendering to about 10 fps with something like requestAnimationFrame, and the caption stays smooth even when interim results come in 20 times a second.

In practice, as an indie developer I tune this in three steps:

1. Measure the interval between interim results (around 50 ms per update if they arrive ~20 times a second).
2. Count how many updates a tail correction takes to converge, and set holdMs a little longer than that span. I recommend starting at 100-150 ms.
3. Throttle rendering to ~10 fps and confirm, on production-grade audio, that the committed display never regresses. Skip this and you meet the rare backward-rewrite pitfall in production instead.

That throttling alone cut caption repaint work by about 50% versus the naive binding. One caveat: shortening holdMs too aggressively pulls corrections into the committed text, so when in doubt, err longer to avoid it.

A guard against rewinding committed text

Occasionally the model re-translates past the committed boundary. Leave this unguarded and your carefully settled opening moves, erasing the whole point of stabilization.

The code treats committed as an anchor and handles a hypothesis that doesn't start with it in two tiers. When the divergence is small (a few characters), it takes it as a legitimate correction and shrinks committed back to the longest common prefix. When the divergence is large, it ignores the hypothesis's regression and treats it as committed plus an appended tail — assuming "the model briefly re-translated a lot but will catch up." In practice, just carrying this rewind budget made the committed display almost stop jumping.

If the Live API session itself drops and the hypothesis stream breaks, handle that on the reconnect and session resumption side, not in the caption layer. The stabilizer keeps committed intact, so after reconnecting you can resume drawing from the volatile continuation.

Wrapping up: the next step

What I did is simple. Stop treating the translation as one string; split it into a committed prefix and a volatile tail, and slip in one guard that forbids backward rewrites. That alone stopped the Live Translate captions from dancing and turned them into something that settles quietly from the front.

If I try one more thing next, it would be closing committed earlier by detecting sentence breaks — punctuation or silent gaps — even before isFinal arrives. Landing the commit moments on the pauses in speech should sit more naturally with how a reader breathes.

These are small refinements, but I hope they help with your own implementation. Thank you for reading.