◈ API / SDK/2026-07-30Advanced

What Decided Our Cascade's Economics Wasn't the Escalation Rate

Putting Flash-Lite in front of 3.6 Flash and escalating only the hard items looks like an easy win. Estimating it from the escalation rate alone will mislead you. Here is the break-even solved with the escalated subset's output length in the equation.

Gemini API¹⁹⁹ Flash-Lite² Cost Modeling Model Routing² Indie Development¹³

✦ Premium Article

The night Gemini 3.5 Flash-Lite pricing landed, I rebuilt my estimate sheet. Input $0.30, output $2.50 per million tokens. Next to the Gemini 3.6 Flash pricing I run on daily — $1.50 input, $7.50 output — that is 5× cheaper on input and 3× cheaper on output.

As an indie developer, the workload I care about is a batch job that writes descriptions and categories for wallpaper images. Run everything through Flash-Lite, escalate only the questionable items to 3.6 Flash. My estimate said the cascade would cut total spend nearly in half. I assumed a 25% escalation rate.

When I actually built it, the savings came in far below the estimate.

The escalation rate was not the problem. The escalated items produced longer outputs than the population average. Obvious in hindsight, and completely absent from my equation.

Write the cascade cost down first

Per item, the cheap stage runs on everything and the strong stage runs on the escalated fraction e.

from dataclasses import dataclass
 
@dataclass(frozen=True)
class Price:
    name: str
    inp: float   # USD / 1M input tokens
    out: float   # USD / 1M output tokens
 
LITE  = Price("gemini-3.5-flash-lite", 0.30, 2.50)
FLASH = Price("gemini-3.6-flash",      1.50, 7.50)
 
def unit_cost(p: Price, t_in: float, t_out: float) -> float:
    """Cost of a single call in USD."""
    if t_in < 0 or t_out < 0:
        raise ValueError(f"token counts must be non-negative: {t_in=}, {t_out=}")
    return (t_in * p.inp + t_out * p.out) / 1_000_000
 
def cascade_unit_cost(t_in, t_out_lite, t_out_flash, e, k=1.0) -> float:
    """Per-item cascade cost.
    e: escalation rate, k: output-length multiplier of the escalated subset
    """
    if not 0.0 <= e <= 1.0:
        raise ValueError(f"escalation rate out of range: {e}")
    if e * k > 1.0:
        # If the escalated subset is k times longer, e*k <= 1 must hold
        # for the population mean to stay consistent.
        raise ValueError(f"inconsistent bias: e*k must be <= 1 (got {e * k:.3f})")
    first  = unit_cost(LITE, t_in, t_out_lite)
    second = unit_cost(FLASH, t_in, t_out_flash * k)
    return first + e * second

Escalated items pay for input twice — once at the cheap stage, once at the strong stage. This is the part everyone worries about in a cascade, and it is where I started looking too.

The escalated subset is longer

Items escalate because the cheap stage could not settle them: ambiguous images, crowded compositions, subjects that do not fit the instruction cleanly. Hand those to a stronger model and the response comes back longer. With a reasoning model, the thinking side grows as well.

So I put the escalated subset's output length into the equation as k times the population mean. k is a measured value. Setting it to 1.0 assumes escalated items return at average length, and that is the assumption I had been making without noticing.

One constraint comes with it: e * k <= 1 must hold for the population mean to stay consistent. The implementation above raises on violations. When you edit an estimate by hand, it is easy to cross that line and still get a number out.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦A break-even equation that includes the escalated subset's output-length multiplier k, plus a runnable implementation

✦For an input-dominant workload (88.9% of spend is input), k=2 moves the break-even by only 10.0%, while an output-dominant one drops 48.8%

✦The sample size needed to pin the escalation rate to within 4 points, and how a 0.5 error in k swings savings by 12 points

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Solve for the break-even

At what escalation rate does the cascade cost the same as the strong model alone?

def breakeven_e(t_in, t_out_lite, t_out_flash, k=1.0) -> float:
    """Escalation rate at which the cascade matches the strong model alone."""
    strong = unit_cost(FLASH, t_in, t_out_flash)
    first  = unit_cost(LITE,  t_in, t_out_lite)
    second = unit_cost(FLASH, t_in, t_out_flash * k)
    if second <= 0:
        return float("inf")
    return (strong - first) / second

Solved for three workload shapes. Input-dominant is one image plus a short instruction (1,600 in / 40 out), balanced is summarization or extraction (2,000 / 800), output-dominant is generation with reasoning (600 / 2,400).

Workload	k=1.0	k=1.5	k=2.0	k=3.0
Input-dominant (one image + short instruction)	78.5%	74.4%	70.7%	64.2%
Balanced (summarize, extract)	71.1%	53.3%	42.7%	30.5%
Output-dominant (generate, reason)	67.3%	45.6%	34.5%	23.2%

For an output-dominant workload, moving k from 1.0 to 2.0 cuts the tolerable escalation rate from 67.3% to 34.5% — roughly half. My confidence that "25% leaves plenty of headroom" came from reading only the k=1.0 column.

Holding the escalation rate at 25%, here is what savings look like per million items.

Workload	Strong only	Cascade (k=1)	Cascade (k=2)	Savings k=1	Savings k=2
Input-dominant	$2,700	$1,255	$1,330	53.5%	50.7%
Balanced	$9,000	$4,850	$6,350	46.1%	29.4%
Output-dominant	$18,900	$10,905	$15,405	42.3%	18.5%

Same escalation rate, same price table. Only k differs — and for the output-dominant workload it takes savings from 42.3% down to 18.5%.

The part that ran opposite to my prediction

My hypothesis going in: escalated items pay input twice, so the heavier the input, the worse a cascade performs.

The arithmetic said the opposite.

Workload	Input share of spend	Break-even k=1	Break-even k=2	Drop in break-even
Input-dominant	88.9%	78.5%	70.7%	10.0%
Balanced	33.3%	71.1%	42.7%	40.0%
Output-dominant	4.8%	67.3%	34.5%	48.8%

Where input is 90% of spend, doubling k moves the break-even by only 10.0%. Where output dominates, it moves 48.8%.

The reason is the direction of the price gap. Input is 5× cheaper on the small model, so the first stage's input costs 20% of the strong model's. Paying twice gives 0.30 + 1.50 = 1.80 against 1.50 for the strong model alone — an increment of 0.30. Small. But k applies only to the output side. When output dominates spend, k passes through almost undiminished.

Double-billed input and biased output length are risks for different workloads. Input-heavy pipelines survive the double billing. Output-heavy pipelines break on the bias, not the double billing.

I could not see that inversion until I wrote the equation and pushed numbers through it. Staring at a price table gives you no feel for which term is doing the work.

How much a wrong k costs

k is an estimate, so here is the sensitivity (output-dominant, e fixed at 0.25).

k	Spend per 1M items	Savings
1.00	$10,905	42.3%
1.25	$12,030	36.3%
1.50	$13,155	30.4%
2.00	$15,405	18.5%
2.50	$17,655	6.6%
3.00	$19,905	-5.3%

At k=3.0 savings go negative: the cascade costs more than the strong model alone. Missing k by the gap between 2.0 and 2.5 swings savings by 12 points.

Paying for the gate turned out to be second-order

Should the escalation decision be its own call, or should the cheap stage report a confidence score inline? I went back and forth on this for a while, because sending the input again just to decide felt wasteful.

Here are the savings with a Flash-Lite gate call on every item (output-dominant, k=2.0).

Gate configuration	e=0.10	e=0.25	e=0.40
No gate call	47.8%	18.5%	-10.8%
0 output tokens (input resend only)	46.8%	17.5%	-11.7%
30 output tokens	46.4%	17.1%	-12.1%
120 output tokens	45.2%	16.0%	-13.3%
300 output tokens	42.9%	13.6%	-15.7%

No gate versus a 30-token gate is a 1.4-point difference. Even a 300-token gate costs 4.9 points. Moving the escalation rate from 0.10 to 0.40 costs 58 points. A separate gate call resends the input, so even at zero output tokens it runs $180 per million items — still a different order of magnitude.

Saving on gate cost at the expense of gate accuracy was optimizing the wrong term. I changed the design here: the inline confidence score is gone, and the decision lives in its own small call.

When production escalation runs past your estimate

Calculating a break-even does not stop the escalation rate from spiking in production. On a day when the input distribution shifted — a run of images with fine-grained composition — escalation came in at roughly double my estimate. That day's spend beat the strong model alone.

My fix is a per-window escalation cap. Once the cap is hit, remaining items keep the cheap-stage result and get a needs_review flag so the next window can pick them up.

The gotcha is implementing that cap as a silent drop. Without the flag, the quality regression shows up in neither the invoice nor the logs.

class EscalationBudget:
    """Per-window escalation cap; overflow defers to the next window."""
 
    def __init__(self, window_items: int, max_rate: float):
        if not 0.0 < max_rate <= 1.0:
            raise ValueError(f"max_rate out of range: {max_rate}")
        self.cap = max(1, int(window_items * max_rate))
        self.used = 0
        self.deferred: list[str] = []
 
    def allow(self, item_id: str) -> bool:
        if self.used < self.cap:
            self.used += 1
            return True
        self.deferred.append(item_id)   # never drop silently
        return False

I recommend setting the cap somewhat inside the computed break-even. Mine sits at 70% of it. Savings may land below the estimate, but the cascade never ends up costing more than the single-model setup.

Only two numbers need measuring before you switch

e and k. Neither is observable from the cheap stage alone, because you cannot measure the strong stage's output length without running the strong stage.

So run a small paired sample first. Send the same inputs to both models and record per-item output tokens alongside the gate decision.

import random
import statistics
from typing import Iterable, NamedTuple
 
class Paired(NamedTuple):
    item_id: str
    escalated: bool       # did the gate mark this for escalation
    out_tokens_flash: int # strong-stage output tokens, from usage_metadata
 
def estimate_e_and_k(rows: Iterable[Paired]) -> dict:
    rows = list(rows)
    if not rows:
        raise ValueError("no paired rows collected")
    esc = [r.out_tokens_flash for r in rows if r.escalated]
    mean_all = statistics.fmean(r.out_tokens_flash for r in rows)
    if mean_all <= 0:
        raise ValueError("mean output tokens is zero; check usage_metadata parsing")
    e = len(esc) / len(rows)
    k = (statistics.fmean(esc) / mean_all) if esc else 1.0
    return {"n": len(rows), "e": e, "k": k, "mean_all": mean_all}
 
def bootstrap_k(rows: list[Paired], iters: int = 2000, seed: int = 17) -> tuple:
    """95% interval for k. A small escalated subset widens it honestly."""
    rng = random.Random(seed)
    ks = []
    for _ in range(iters):
        sample = [rows[rng.randrange(len(rows))] for _ in range(len(rows))]
        try:
            ks.append(estimate_e_and_k(sample)["k"])
        except ValueError:
            continue
    if len(ks) < iters * 0.5:
        raise RuntimeError("bootstrap unstable; collect more paired rows")
    ks.sort()
    return ks[int(0.025 * len(ks))], ks[int(0.975 * len(ks))]

For sample sizing, here are Wilson intervals assuming an escalation rate near 0.25.

Sample size	95% interval	Interval width
50	15.1% – 38.5%	23.4 points
100	17.5% – 34.3%	16.8 points
200	19.5% – 31.4%	11.9 points
400	21.0% – 29.5%	8.5 points
800	22.1% – 28.1%	6.0 points

At 50 samples the interval spans 23.4 points, which says nothing useful about headroom against a 34.5% break-even. At 400 you get roughly 4 points per side. k is worse off, since only escalated items contribute — at 400 paired rows the effective sample is around 100. I always report the bootstrap interval alongside the point estimate.

Why I stopped using self-reported confidence as the gate

Asking the cheap stage for a 0-to-1 confidence score was my first attempt, and I dropped it. Most of the misclassifications came back with high confidence. The premise that uncertain items score low simply did not hold for my subject matter.

What I run instead: sample the cheap stage twice and escalate when the two results disagree. No separate judge model, just double the cheap-stage cost.

Whether that doubling is affordable comes out of the same equation (e=0.25, k=2.0).

Workload	Savings, one cheap call	Savings, two cheap calls	Break-even, two cheap calls
Input-dominant	50.7%	29.3%	51.3%
Balanced	29.4%	0.6%	25.3%
Output-dominant	18.5%	-14.2%	17.7%

An input-dominant workload keeps 29.3% savings even with the cheap stage run twice. Balanced drops to 0.6%, which makes the whole arrangement pointless. My wallpaper classification sits on the input-dominant side, so double sampling was available to me. I have not carried the same gate over to the summarization pipeline.

Applying this to your own workload

Three steps.

Measure input and output tokens on even a single real item, and compute what share of spend is input
Run a small paired sample to measure e and k. k is the escalated subset's mean output length divided by the population mean
Feed k into breakeven_e, then compare the result against your measured e

If input is more than 80% of spend, a cascade is structurally forgiving about escalation-rate drift. If output dominates, switching without measuring k first is a gamble.

A 5× gap in unit price is not a 5× gap in total spend. The multipliers apply to different terms and to different subsets. I mixed those two together, built an estimate on it, and rebuilt it. If this saves you one round of that, it has done its job. Thank you for reading.

Prices quoted here were verified on July 30, 2026. They do get revised, so confirm against primary sources before running your own numbers.

Thank You for Reading

Gemini Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.