Before Free Users Quietly Eat Your Margin: Tier Design and Cost Ceilings for Gemini API Apps

Accrue measured cost, not estimates

Prices get revised. Write a fixed unit price into an article and it may be stale by the time you read it. So on the design side, escape the unit price into a single constant, multiply it by measured tokens, and accrue. Always confirm the latest rate on the official pricing page and swap only this constant.

# USD per million tokens. ALWAYS update to the official current value.
# Differs by model and by input/output. This is the single point you swap.
PRICE = {
    "gemini-3.1-flash-lite": {"in": 0.10, "out": 0.40},
    "gemini-3.5-flash":      {"in": 0.30, "out": 2.50},
    "gemini-3.1-pro":        {"in": 1.25, "out": 10.00},
}
 
def estimate_cost_usd(model: str, in_tok: int, out_tok: int) -> float:
    p = PRICE[model]
    return (in_tok * p["in"] + out_tok * p["out"]) / 1_000_000

Call estimate_cost_usd per request and accrue it per user per month. Because you accrue dollars, not counts, light questions stay far from the ceiling while heavy work approaches it fast — behavior that matches intuition. The classic accident, a few heavy users melting your margin, gets contained naturally.

Judge the monthly cost ceiling at the edge

Do the judging before the request reaches origin, at the Cloudflare Workers edge. Block abuse and overuse before they touch the Gemini API. If it never arrives, the cost is zero. The only difference from a rate limit is that what you accrue in KV is USD, not counts.

interface Budget {
  spentUsd: number;   // accumulated cost this month
  resetAt: number;    // next-month reset, Unix(ms)
}
interface Env { BUDGET_KV: KVNamespace; }
 
// Per-tier monthly cost ceiling (USD). Back it out from price x expected use.
const COST_CEILING: Record<string, number> = {
  free: 0.15,   // for free you control the deficit directly
  pro: 3.00,    // ceiling that still leaves margin within the subscription
  premium: 12.00,
};
 
async function withinBudget(
  env: Env, userId: string, tier: string
): Promise<{ ok: boolean; spent: number; ceiling: number }> {
  const ceiling = COST_CEILING[tier] ?? COST_CEILING.free;
  const key = `budget:${userId}`;
  const now = Date.now();
  const b = (await env.BUDGET_KV.get(key, "json")) as Budget | null;
 
  // Start from 0 if there's no current month or we're past the reset.
  const cur = b && b.resetAt > now ? b : { spentUsd: 0, resetAt: endOfMonth(now) };
  return { ok: cur.spentUsd < ceiling, spent: cur.spentUsd, ceiling };
}
 
// Add real cost after the response (pass the value derived from usage_metadata).
async function addSpend(env: Env, userId: string, costUsd: number) {
  const key = `budget:${userId}`;
  const now = Date.now();
  const b = (await env.BUDGET_KV.get(key, "json")) as Budget | null;
  const cur = b && b.resetAt > now ? b : { spentUsd: 0, resetAt: endOfMonth(now) };
  cur.spentUsd += costUsd;
  await env.BUDGET_KV.put(key, JSON.stringify(cur), {
    expirationTtl: Math.ceil((cur.resetAt - now) / 1000) + 86400,
  });
}
 
function endOfMonth(now: number): number {
  const d = new Date(now);
  return Date.UTC(d.getUTCFullYear(), d.getUTCMonth() + 1, 1);
}

For a user who hits the ceiling, return an upgrade path, not an error. A 429 that says "you've used this month's free allowance — go Pro to continue" is a stronger conversion driver the lower the remaining balance. The defensive mechanism becomes the entry to revenue.

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    const { userId, tier } = await verifyJwt(req); // implementation omitted
    const { ok, spent, ceiling } = await withinBudget(env, userId, tier);
    if (!ok) {
      return new Response(JSON.stringify({
        error: "budget_exceeded",
        message: "Monthly limit reached. Upgrade to continue.",
        spent, ceiling,
      }), { status: 429, headers: { "Content-Type": "application/json" } });
    }
    // Call Gemini here, then accrue the returned real cost via addSpend.
    return new Response("...", { status: 200 });
  },
};

KV is eventually consistent, so under very high concurrency a ceiling can be exceeded slightly. I run a two-step guard — soft warning at ceiling x 0.95, hard cut at x 1.1 — and tolerate small overruns. The goal is preventing accidents, not penny-exact accounting.

Three quiet leaks that drain margin

Tiers and a cost ceiling hold the broad line. What remains are leaks that grow tokens where you don't look. Here they are in the order I actually plugged them.

Leak 1: a long system prompt on the free tier. The system prompt is a fixed cost that rides on every request's input. Use a careful 2,000-token instruction for all users and the pure fixed cost swells as requests grow. Trim free to under 300 tokens and create the quality gap with the paid tier's detailed prompt — push the party that pays the fixed cost toward the layer that has revenue.

Leak 2: sending the whole chat history every time. Resend the full history each turn and input tokens grow linearly as the conversation lengthens. At 50 turns a single request's input hitting tens of thousands of tokens is routine. Keep only the last N turns in detail and compress earlier ones into a summary — a sliding window keeps tokens within a fixed band.

def build_window(messages: list[dict], recent_turns: int = 8, summary: str = "") -> list[dict]:
    """Keep only the last recent_turns turns; replace earlier ones with a summary."""
    start = len(messages) - recent_turns * 2  # one turn = two messages
    if start <= 0:
        return messages
    recent = messages[start:]
    if not summary:
        return recent
    head = {"role": "user", "parts": [{"text": f"[Summary so far] {summary}"}]}
    return [head] + recent

Leak 3: unbounded retries on error. Retries without exponential backoff turn a single user action into 20-30 billable requests on a transient 503. Always add backoff, a retry cap, and a strict allow-list of which error types are even retryable.

import asyncio, random
from google.api_core.exceptions import ResourceExhausted, ServiceUnavailable
 
async def call_with_backoff(coro_factory, max_retries: int = 3):
    last = None
    for attempt in range(max_retries):
        try:
            return await coro_factory()
        except (ResourceExhausted, ServiceUnavailable) as e:
            last = e
            if attempt == max_retries - 1:
                break
            await asyncio.sleep(2 ** attempt + random.uniform(0, 1))  # with jitter
        except Exception as e:
            raise RuntimeError(f"non-retryable error: {e}") from e  # stop immediately
    raise RuntimeError(f"retry cap exceeded: {last}")

Cost sets the price floor

The price of an app with AI features is set not by competitors but by API cost. Take the per-user monthly API cost, add store fees and infrastructure, and leave margin on top. As a rule of thumb, placing the floor at 10-15x the expected cost survives the swings in conversion. At $3/user/month of cost, the monthly floor is roughly $30-45.

That formula holds only because the tier design comes first — containing free with Flash-Lite and covering the paid layer's cost from revenue with Pro. Put everyone on Pro and the margin vanishes at the same price. Price is only the result that sits on top of the cost structure.

Put cost first in the design

Most failed AI monetizations start by deferring cost. Build the feature first, figure out billing later — that order fits poorly with a structure where money leaves on every request.

Reverse it. Decide the tier-to-model mapping first, accrue real cost via usage_metadata, draw the monthly ceiling at the edge. Then polish the free experience to feed conversion. Build the defense, then add the offense. That order alone turns growth toward profit rather than loss. As a first step, compute, just once and with measured rates, what your app costs per month if a thousand people use it daily. Start the design from there and the pain of rebuilding later shrinks a great deal.