◈ API / SDK/2026-05-19Advanced

Wiring Circuit Breakers and Graceful Degradation into Gemini API — an Indie App's Stability-First Notes

When you run Gemini API in production for an indie app, something upstream breaks at least a few times a month. Here are the building blocks for circuit breakers and graceful degradation I settled on, with the implementation traps I actually hit.

Gemini API¹⁹² Circuit Breaker² Availability Production³² Architecture⁹

✦ Premium Article

Around 2 AM on a night in late March 2026, a phone notification told me my crash rate had spiked. I opened Crashlytics and traced it to a chain of 503 responses from the Gemini API, followed by the main thread dutifully waiting out a 30-second timeout. AdMob impressions sagged in real time; that night alone, ad revenue ended up at less than half my usual figure. The app itself was fine, but a few minutes of upstream weather had punched a hole in the daily revenue. The next morning I sat down and rebuilt the circuit breaker and graceful degradation design more seriously. This piece is my notes from that rebuild and the tuning I have done since.

Why an Indie App Should Stop Aiming for "Always Working"

I have been shipping smartphone wallpaper apps and visualization apps as an indie developer since 2014, and the catalog has now passed 50 million cumulative downloads. As an indie developer Hirokawa, operations are basically a team of one. Even with Cloudflare Workers and CDN layers in front, an upstream provider like Gemini API will eventually have a few minutes that ripple through to the user.

So I gave up on a different goal. Instead of "Gemini API always answers", I aim for "the app stays quietly useful even in the seconds Gemini API does not answer". A nominal 99.9% availability sounds great, but it leaves you about 43 minutes of allowed downtime per month, and for an indie developer that 43 minutes is almost guaranteed to land on the weeknight evening when ads are actually clicking. The thing worth defending is not perfect uptime, it is the user experience during those minutes.

Circuit breakers and graceful degradation are the concrete tools for that mindset. Over 14 months since I shipped this design, upstream-triggered fatal crashes across my three main apps fell by 87% in Crashlytics, and the depth of revenue dips during incidents shrank noticeably.

Minimum Viable Circuit Breaker — Skeleton First

Plenty of libraries exist, but for an indie app the best insurance is "code I can fully explain to myself in one sitting." Here is the minimum Python skeleton I run server-side. While request volume is modest this is enough on its own.

import time
from dataclasses import dataclass, field
from enum import Enum
from threading import Lock
 
class State(str, Enum):
    CLOSED = "closed"
    OPEN = "open"
    HALF_OPEN = "half_open"
 
@dataclass
class Breaker:
    failure_threshold: int = 5          # consecutive failures to OPEN
    recovery_seconds: float = 30.0      # OPEN until HALF_OPEN
    half_open_max_calls: int = 3        # trial calls allowed
    state: State = State.CLOSED
    fail_count: int = 0
    opened_at: float = 0.0
    half_open_calls: int = 0
    lock: Lock = field(default_factory=Lock)
 
    def allow_request(self) -> bool:
        with self.lock:
            if self.state == State.OPEN:
                if time.time() - self.opened_at >= self.recovery_seconds:
                    self.state = State.HALF_OPEN
                    self.half_open_calls = 0
                else:
                    return False
            if self.state == State.HALF_OPEN:
                if self.half_open_calls >= self.half_open_max_calls:
                    return False
                self.half_open_calls += 1
            return True
 
    def on_success(self) -> None:
        with self.lock:
            self.fail_count = 0
            self.state = State.CLOSED
            self.half_open_calls = 0
 
    def on_failure(self) -> None:
        with self.lock:
            self.fail_count += 1
            if self.state == State.HALF_OPEN or self.fail_count >= self.failure_threshold:
                self.state = State.OPEN
                self.opened_at = time.time()
                self.fail_count = 0

The skeleton is under 60 lines. What matters in production is choosing failure_threshold and recovery_seconds against your real traffic shape. My wallpaper app backend runs around 4–12 req/s, so 5 consecutive failures (about a second of bad weather) into OPEN, with a 30-second cooldown, is my current resting point.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦How to pick Closed/Open/Half-Open thresholds for an indie app's traffic shape

✦A four-stage fallback (Gemini 3 Pro → 2.5 Flash → cache → default text) with working code

✦A retry budget pattern that protects p95 latency instead of inflating it

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Picking Closed / Open / Half-Open Thresholds

The hardest number was the Half-Open concurrent trial count. One trial recovers slowly, five trials throw a tiny load spike at an API that has not actually recovered yet, and you flip back to OPEN. Three trials landed as the right compromise: three independent successes is a stiff enough recovery test, but not so demanding that healthy minutes feel locked out.

A second trap is running a single breaker for the whole API. Gemini API behaves differently per endpoint (generateContent vs embedContent) and per model family (Flash vs Pro). I now keep a breaker per "model × endpoint." It is common to see Flash congested while Pro is perfectly fine, and a single global breaker would punish all calls equally.

Four-Stage Fallback — Gemini 3 Pro → 2.5 Flash → Cache → Default

The moment the breaker trips, the user-facing experience is what matters. I aim for two rules: the app must not go silent, and every tap must produce a visible response. Here is the four-stage fallback chain I run.

async def generate_caption(image_id: str, prompt: str) -> str:
    # Step 1: high-quality model
    if pro_breaker.allow_request():
        try:
            return await call_gemini("gemini-3-pro", prompt, timeout=4.0)
        except Exception:
            pro_breaker.on_failure()
        else:
            pro_breaker.on_success()
 
    # Step 2: cheaper / faster model
    if flash_breaker.allow_request():
        try:
            return await call_gemini("gemini-2.5-flash", prompt, timeout=2.0)
        except Exception:
            flash_breaker.on_failure()
        else:
            flash_breaker.on_success()
 
    # Step 3: last-24h cache of similar prompts
    cached = await cache.get_similar(prompt, top_k=1, min_score=0.86)
    if cached:
        return cached.text + "  (cached)"
 
    # Step 4: built-in default — never stop the experience
    return DEFAULT_CAPTIONS_BY_CATEGORY.get(image_id, "A piece for quiet appreciation")

The fourth stage is the indie-friendly trick: I ship 30–50 natural short captions per category inside the app and pick deterministically by image_id hash. The user sees "a slightly plain caption," not "the server is down." Replacing the empty-string path with this default alone cut several non-fatal exception categories in Crashlytics dramatically.

Retry Budgets That Protect p95 Latency

Pairing the breaker with a per-request retry budget made a bigger difference than the breaker itself. A naive exponential backoff with five retries inflates the wait time the user feels before the breaker flips, and p95 latency goes up sharply. Here is the budget pattern I use.

async def call_with_budget(model: str, prompt: str, *, deadline_ms: int = 1500):
    start = time.monotonic()
    delays = [0.0, 0.15, 0.40]  # up to 3 attempts
    last_exc = None
    for i, d in enumerate(delays):
        elapsed_ms = (time.monotonic() - start) * 1000
        if elapsed_ms + d * 1000 > deadline_ms:
            break  # giving up keeps p95 stable
        if d:
            await asyncio.sleep(d + random.uniform(0, 0.05))  # full-jitter-ish
        try:
            return await call_gemini(model, prompt, timeout=0.9)
        except RetryableError as e:
            last_exc = e
            continue
    raise last_exc

The point is that deadline_ms caps the whole call from above, and any retry that would overshoot is simply skipped. After I rolled this out, p95 across the main apps settled near 1.8 seconds. More retries always lift success rate on paper, but the user-experience cost is real. For indie use, my taste is three attempts within a 1.5-second budget.

Three Metrics That Are Enough — Failure Rate, p95, Fallback Rate

Cutting the dashboard down keeps operations sane for a one-person team. I look at three numbers daily. Failure rate paged when it crosses 1% on a one-minute window. p95 paged when it crosses 2.5 seconds on a five-minute average. Fallback rate (the share of calls that fell to stages 2–4) flagged for the weekly review when it exceeds 8% of the day's total. Three metrics miss things, but it is the limit of what I can actually act on.

A side benefit: watching fallback rate let me measure how the Pro/Flash quality gap moves ad CTR. When traffic falls to Flash, captions become a touch plainer, ad tap-through edges up slightly, and the click rate into in-app purchases dips a fraction. Whether your revenue leans ad-heavy or purchase-heavy informs which model you should run as the first stage.

Implementation Pitfalls and the Recommendations That Stuck

A few traps almost broke this design on me weeks after I shipped it. Copying the skeleton above is not enough without the points below.

First, layered timeouts. There are usually five timeouts stacked end-to-end: HTTP client, retry budget, breaker decision, runtime (Cloud Run etc.), and the CDN cap. If a lower layer is longer than an upper layer the breaker never fires; if an upper layer is shorter, the runtime kills the process before the fallback path runs. My personal rule is "shorter inside": HTTP 0.9s → retry budget 1.5s → breaker decision window 3s → runtime 6s.

Second, idempotency. Gemini API calls can have side effects on the server side, particularly when Function Calling triggers a database update. I hash request_id with SHA-1 and memoize the result in KV for ten minutes, so a retry from a flaky client cannot double-write.

Third, observation granularity. Log every breaker state transition to Cloud Logging with a dedicated label. Being able to answer "when did this go OPEN, when did it move to HALF_OPEN?" turns multi-hour debugging sessions into ten-minute ones. I stream those transitions into BigQuery and once a week list every window that stayed OPEN for more than five minutes.

For an indie app, designing around upstream weather has turned out to be one of the highest-leverage investments for both ad revenue and subscription conversion. Giving up on perfect uptime and instead polishing quiet degradation and quick recovery fits my temperament. If you operate something similar on your own, I hope a few of these notes save you a night. Thank you for reading.

Thank You for Reading

Gemini Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.