◈ API / SDK/2026-04-11Advanced

Gemini API Rate Limits and 429 Handling: Operational Notes from an Indie Mobile App

Operational notes on handling Gemini API rate limits and 429 errors in a production indie mobile app: exponential backoff, adaptive control, multi-key pooling, and Cloud Monitoring integration, all rebuilt after a real incident.

gemini-api²⁸⁶ rate-limiting⁴quota-managementpython¹⁰⁸ production¹⁴⁰ retry⁶ monitoring⁵

✦ Premium Article

The Night a Batch Job Went Quiet for Thirty Minutes

The backend of a wallpaper app I run as a solo developer leans on Gemini API for image captions, tag completion, and review summarisation. It is a small service funded by ad revenue, which means an API stall is not an abstraction — it is the day's work stopping.

The rate-limiting code I write today exists because of one specific bad night. A scheduled job was supposed to regenerate metadata tags for about 80,000 wallpapers. I had retries, but exponential backoff was attached at the job level, not per request. The first key that hit 429 RESOURCE_EXHAUSTED got hammered by retries from every concurrent worker, the API went silent for more than thirty minutes, and the App Store Connect review work I had lined up for the morning slipped a full day. Since then, rate limiting in this app is treated as foundational, not as a "we'll harden it later" item.

What follows is the operational shape of that hardening: how Gemini API's quota system actually behaves under load, how I run exponential backoff with jitter, what adaptive rate control buys you, how I split keys via Vertex AI service accounts, and how Cloud Monitoring stitches the alerts together. Every code block is something currently running in the wallpaper app's Cloud Functions / Cloud Run backend, with business specifics scrubbed.

Who this is for

Engineers who have just put Gemini API into a real service and have seen their first few 429s or TPM trips
Indie or small-team developers who want a Cloud Monitoring setup that fits on the back of a napkin
Anyone trying to draw clean cost/quality lines for AI on top of an ad-funded app economy

Understanding the Gemini API Quota System

Before you can manage quotas effectively, you need a precise understanding of how they work. Gemini API enforces limits across three distinct dimensions.

The Three Quota Dimensions

1. RPM (Requests Per Minute)

This is the limit you'll hit most often. It varies by model and billing plan. On the free tier, Gemini 2.5 Pro allows just 5 RPM. Paid plans can reach 1,000 RPM or more.

2. TPM (Tokens Per Minute)

Rather than counting requests, this limit tracks token throughput. If you're passing large contexts or documents, you may hit your TPM limit even before your RPM limit.

3. RPD (Requests Per Day)

A daily ceiling. Free-tier limits are strict; paid plans are far more generous.

How to Check Your Current Limits

You can view your current quotas in Google AI Studio under "Get API Key," or in the Google Cloud Console under "Quotas & System Limits." You can also track token consumption per-request through the API response metadata.

import google.generativeai as genai
 
genai.configure(api_key="YOUR_GEMINI_API_KEY")
 
model = genai.GenerativeModel("gemini-2.5-pro")
response = model.generate_content("Hello, production world!")
 
# usage_metadata gives you per-request token breakdown
print(f"Input tokens:  {response.usage_metadata.prompt_token_count}")
print(f"Output tokens: {response.usage_metadata.candidates_token_count}")
print(f"Total tokens:  {response.usage_metadata.total_token_count}")

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Postmortem from a wallpaper app's 80,000-item nightly batch where a 429 cascade froze the API for 30+ minutes, with the exact retry, jitter and timeout numbers I rebuilt around

✦Seven implementation insights you will not find in the official docs (usage_metadata lag, Retry-After flakiness, IAM-based multi-key pooling, TPM vs RPM disambiguation, and more)

✦Decision lines for splitting Flash vs Pro in an AdMob-funded mobile app and a daily Cloud Billing alert formula (monthly_budget / 30 * 1.3) you can copy directly

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Classifying Rate Limit Errors

Not all 429 errors are the same. Each subtype calls for a different response strategy.

Error Subtypes and What They Mean

Pattern A: RPM Exceeded (most common)

google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for quota metric
'generate_requests_per_minute'

Fix: Slow down your request rate or use a request queue.

Pattern B: TPM Exceeded

google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for quota metric
'generate_tokens_per_minute'

Fix: Reduce context size or decrease request frequency.

Pattern C: Concurrent Request Limit

google.api_core.exceptions.ResourceExhausted: 429 Too many concurrent requests

Fix: Use a semaphore to cap simultaneous in-flight requests.

Error Classification Helper

from google.api_core.exceptions import ResourceExhausted
 
def classify_rate_limit_error(error: ResourceExhausted) -> str:
    """Identify which quota type caused a 429 error."""
    message = str(error)
 
    if "generate_requests_per_minute" in message or "per_minute_per_project" in message:
        return "RPM_EXCEEDED"
    elif "tokens_per_minute" in message or "generate_tokens" in message:
        return "TPM_EXCEEDED"
    elif "concurrent" in message.lower():
        return "CONCURRENT_EXCEEDED"
    else:
        return "UNKNOWN_QUOTA"
 
try:
    response = model.generate_content(prompt)
except ResourceExhausted as e:
    error_type = classify_rate_limit_error(e)
    print(f"Error type: {error_type}")
    # Branch to the right strategy based on error_type

Exponential Backoff with Jitter

Once you know you're being rate-limited, the next question is: how do you retry? A naive fixed-interval retry creates the thundering herd problem—every client retries at the same moment, hammering the API all over again. The solution is exponential backoff with jitter (random delay spread).

Full Jitter Implementation

import time
import random
import functools
from typing import Callable, TypeVar, Any
from google.api_core.exceptions import ResourceExhausted, ServiceUnavailable
 
T = TypeVar('T')
 
def exponential_backoff_with_jitter(
    max_retries: int = 5,
    base_delay: float = 1.0,
    max_delay: float = 60.0,
    jitter_type: str = "full"  # "full" or "decorrelated"
) -> Callable:
    """
    Decorator that retries a function using exponential backoff + jitter.
 
    Args:
        max_retries: Maximum number of retry attempts.
        base_delay:  Initial delay in seconds.
        max_delay:   Maximum delay cap in seconds.
        jitter_type: "full" (AWS-recommended) or "decorrelated" (smoother spread).
    """
    def decorator(func: Callable[..., T]) -> Callable[..., T]:
        @functools.wraps(func)
        def wrapper(*args: Any, **kwargs: Any) -> T:
            last_sleep = base_delay
 
            for attempt in range(max_retries + 1):
                try:
                    return func(*args, **kwargs)
 
                except (ResourceExhausted, ServiceUnavailable) as e:
                    if attempt == max_retries:
                        raise  # Re-raise after exhausting retries
 
                    if jitter_type == "full":
                        # Full Jitter: uniform random between 0 and capped exponential
                        sleep_time = random.uniform(
                            0,
                            min(max_delay, base_delay * (2 ** attempt))
                        )
                    elif jitter_type == "decorrelated":
                        # Decorrelated Jitter: less bursty, smoother distribution
                        sleep_time = min(
                            max_delay,
                            random.uniform(base_delay, last_sleep * 3)
                        )
                        last_sleep = sleep_time
                    else:
                        sleep_time = base_delay * (2 ** attempt)
 
                    print(
                        f"Rate limited (attempt {attempt + 1}/{max_retries}). "
                        f"Retrying in {sleep_time:.2f}s. Error: {e}"
                    )
                    time.sleep(sleep_time)
 
        return wrapper
    return decorator
 
# Usage
@exponential_backoff_with_jitter(max_retries=5, base_delay=2.0, max_delay=60.0)
def generate_with_retry(model, prompt: str) -> str:
    response = model.generate_content(prompt)
    return response.text
 
model = genai.GenerativeModel("gemini-2.5-flash")
try:
    result = generate_with_retry(model, "Explain production API design best practices.")
    print(result)
except ResourceExhausted:
    print("Max retries exceeded. Try again later.")

Respecting the `Retry-After` Header

Some API responses include a Retry-After hint. Use it when available to avoid unnecessary waiting.

def get_retry_after_seconds(error: Exception) -> float | None:
    """Extract Retry-After value from error metadata, if present."""
    if hasattr(error, 'metadata'):
        for key, value in error.metadata:
            if key.lower() == 'retry-after':
                try:
                    return float(value)
                except ValueError:
                    pass
    return None

Adaptive Rate Control

Retrying is reactive. Adaptive rate control is proactive—it adjusts your request rate before you get rate-limited, based on observed API behavior. The implementation uses a token bucket algorithm with dynamic adjustment.

Token Bucket with Dynamic Throttling

import threading
import time
from collections import deque
from dataclasses import dataclass
from typing import Optional
 
@dataclass
class RateLimiterConfig:
    rpm_limit: int = 60          # Hard RPM ceiling
    tpm_limit: int = 100_000     # Hard TPM ceiling
    safety_margin: float = 0.85  # Target 85% of limit as a buffer
    window_seconds: int = 60     # Sliding window duration
 
class AdaptiveRateLimiter:
    """
    Adaptive rate limiter using a sliding window.
 
    Automatically slows down on errors and ramps back up on sustained success.
    Designed to stay comfortably under quota while maximizing throughput.
    """
 
    def __init__(self, config: RateLimiterConfig):
        self.config = config
        self._lock = threading.Lock()
        self._request_times: deque = deque()
        self._token_usage: deque = deque()  # (timestamp, token_count)
        self._current_rpm = config.rpm_limit * config.safety_margin
        self._consecutive_errors = 0
        self._consecutive_successes = 0
 
    def _clean_old_records(self):
        """Prune records outside the sliding window."""
        cutoff = time.time() - self.config.window_seconds
        while self._request_times and self._request_times[0] < cutoff:
            self._request_times.popleft()
        while self._token_usage and self._token_usage[0][0] < cutoff:
            self._token_usage.popleft()
 
    def wait_if_needed(self) -> float:
        """
        Block until it's safe to make a request.
        Returns the number of seconds waited.
        """
        with self._lock:
            wait_time = 0.0
            self._clean_old_records()
 
            current_rpm = len(self._request_times)
            if current_rpm >= self._current_rpm:
                oldest = self._request_times[0]
                wait_time = max(
                    0.0,
                    self.config.window_seconds - (time.time() - oldest) + 0.1
                )
 
            if wait_time > 0:
                time.sleep(wait_time)
                self._clean_old_records()
 
            self._request_times.append(time.time())
            return wait_time
 
    def record_success(self, token_count: int = 0):
        """Record a successful request and gradually increase rate."""
        with self._lock:
            if token_count > 0:
                self._token_usage.append((time.time(), token_count))
            self._consecutive_errors = 0
            self._consecutive_successes += 1
            # Every 10 consecutive successes, recover 5% of rate
            if self._consecutive_successes >= 10:
                max_rpm = self.config.rpm_limit * self.config.safety_margin
                self._current_rpm = min(self._current_rpm * 1.05, max_rpm)
                self._consecutive_successes = 0
 
    def record_error(self):
        """Record an error and reduce the effective rate by 20%."""
        with self._lock:
            self._consecutive_errors += 1
            self._consecutive_successes = 0
            self._current_rpm = max(1.0, self._current_rpm * 0.8)
            print(f"⚠️ Rate reduced to {self._current_rpm:.1f} effective RPM")

Multi-API Key Pooling and Load Balancing

When a single API key can't deliver the throughput you need, the next step is to pool multiple keys. Combined with adaptive rate limiting per key, this approach dramatically increases both throughput and resilience.

API Key Pool with Health Checking

import itertools
from typing import List
from dataclasses import dataclass, field
 
@dataclass
class ApiKeyEntry:
    key: str
    limiter: AdaptiveRateLimiter
    error_count: int = 0
    success_count: int = 0
    is_healthy: bool = True
 
class GeminiApiKeyPool:
    """
    Round-robin API key pool with per-key health tracking.
 
    Keys that repeatedly fail are temporarily removed from rotation
    and automatically restored after a cooldown period.
    """
 
    def __init__(self, api_keys: List[str], rpm_per_key: int = 60):
        self._entries: List[ApiKeyEntry] = []
        for key in api_keys:
            config = RateLimiterConfig(rpm_limit=rpm_per_key, safety_margin=0.85)
            self._entries.append(ApiKeyEntry(key=key, limiter=AdaptiveRateLimiter(config)))
        self._cycle = itertools.cycle(self._entries)
        self._lock = threading.Lock()
 
    def get_healthy_key(self) -> Optional[ApiKeyEntry]:
        """Return the next healthy key, or force-reset if all are paused."""
        with self._lock:
            for _ in range(len(self._entries)):
                entry = next(self._cycle)
                if entry.is_healthy:
                    return entry
            # All keys exhausted—force reset
            print("⚠️ All API keys paused. Force-resetting.")
            for e in self._entries:
                e.is_healthy = True
            return next(self._cycle)
 
    def record_key_error(self, entry: ApiKeyEntry):
        """Track errors per key; suspend after 5 consecutive failures."""
        with self._lock:
            entry.error_count += 1
            entry.limiter.record_error()
            if entry.error_count >= 5:
                print(f"🚫 Suspending key ...{entry.key[-4:]} for 30s")
                entry.is_healthy = False
                def restore():
                    time.sleep(30)
                    entry.is_healthy = True
                    entry.error_count = 0
                    print(f"✅ Key ...{entry.key[-4:]} restored")
                threading.Thread(target=restore, daemon=True).start()
 
    def record_key_success(self, entry: ApiKeyEntry, token_count: int = 0):
        entry.success_count += 1
        entry.error_count = max(0, entry.error_count - 1)
        entry.limiter.record_success(token_count)
 
# Usage: load keys from environment variables
import os
 
api_keys = [
    os.environ.get("GEMINI_API_KEY_1", ""),
    os.environ.get("GEMINI_API_KEY_2", ""),
    os.environ.get("GEMINI_API_KEY_3", ""),
]
api_keys = [k for k in api_keys if k]
 
pool = GeminiApiKeyPool(api_keys=api_keys, rpm_per_key=60)
 
def pool_generate(prompt: str, max_retries: int = 3) -> str:
    """High-availability API call with multi-key pooling."""
    for attempt in range(max_retries):
        entry = pool.get_healthy_key()
        if not entry:
            raise RuntimeError("No API keys available")
 
        genai.configure(api_key=entry.key)
        try:
            entry.limiter.wait_if_needed()
            model = genai.GenerativeModel("gemini-2.5-flash")
            response = model.generate_content(prompt)
            pool.record_key_success(entry, response.usage_metadata.total_token_count)
            return response.text
        except ResourceExhausted as e:
            print(f"Attempt {attempt + 1}: Key ...{entry.key[-4:]} rate-limited. Trying another.")
            pool.record_key_error(entry)
            if attempt == max_retries - 1:
                raise
            time.sleep(random.uniform(1, 3))
    raise RuntimeError("All retries failed")

Building a Quota Monitoring and Alert System

Real-time visibility into your API usage is non-negotiable for production systems. Here's how to push custom metrics to Cloud Monitoring and set up proactive alerts.

Sending Custom Metrics to Cloud Monitoring

from google.cloud import monitoring_v3
from google.cloud.monitoring_v3 import types
import datetime
 
class GeminiApiMetricsCollector:
    """
    Pushes Gemini API usage stats to Cloud Monitoring as custom metrics.
 
    Tracks: requests, tokens, errors, and latency per model.
    """
 
    def __init__(self, project_id: str):
        self.project_id = project_id
        self.client = monitoring_v3.MetricServiceClient()
        self.project_name = f"projects/{project_id}"
 
    def _make_time_series(self, metric_type: str, value: float, labels: dict):
        series = types.TimeSeries()
        series.metric.type = f"custom.googleapis.com/{metric_type}"
        for k, v in labels.items():
            series.metric.labels[k] = v
        series.resource.type = "global"
        now = datetime.datetime.utcnow()
        interval = types.TimeInterval(end_time={"seconds": int(now.timestamp())})
        series.points = [types.Point(interval=interval, value={"double_value": value})]
        return series
 
    def record_request(self, model: str, success: bool, token_count: int, latency_ms: float):
        """Push one request's worth of metrics to Cloud Monitoring."""
        labels = {"model": model, "status": "success" if success else "error"}
        series = [
            self._make_time_series("gemini_api/requests_total", 1.0, labels),
            self._make_time_series("gemini_api/latency_ms", latency_ms, labels),
        ]
        if token_count > 0:
            series.append(self._make_time_series("gemini_api/tokens_total", float(token_count), labels))
        if not success:
            series.append(self._make_time_series("gemini_api/errors_total", 1.0, labels))
 
        try:
            self.client.create_time_series(name=self.project_name, time_series=series)
        except Exception as e:
            # Never let monitoring failures bring down the main service
            print(f"⚠️ Metrics push failed (ignored): {e}")

Cost Optimization Strategies

Rate limiting isn't just about staying within quotas—it's also about controlling costs. Here are three high-impact techniques.

1. Context Caching (up to 75% input cost reduction)

When you reuse the same large system prompt across many requests, Context Caching lets you pay roughly 25% of the normal input token price for cached portions.

import google.generativeai as genai
from google.generativeai import caching
import datetime
 
# Cache a large system instruction
cache = caching.CachedContent.create(
    model="gemini-2.5-flash",
    display_name="Shared system prompt",
    system_instruction="You are an expert customer support agent. [long detailed instructions...]",
    ttl=datetime.timedelta(minutes=60),
)
 
# All requests using this model share the cached prompt tokens
model = genai.GenerativeModel.from_cached_content(cached_content=cache)
response = model.generate_content("How do I return product A?")
print(f"Cached tokens (discounted): {response.usage_metadata.cached_content_token_count}")

2. Dynamic Model Selection

Not every request needs your most powerful model. Route by complexity.

def select_model(prompt: str) -> str:
    """Route to the cheapest model that can handle the request."""
    word_count = len(prompt.split())
    needs_reasoning = any(kw in prompt.lower() for kw in ["analyze", "compare", "explain why"])
 
    if word_count < 50 and not needs_reasoning:
        return "gemini-2.5-flash"   # Fast, cheap
    elif word_count < 300:
        return "gemini-2.5-flash"   # Still fine for medium complexity
    else:
        return "gemini-2.5-pro"     # Reserve Pro for genuinely complex tasks

3. Async Batch Processing

import asyncio
from typing import List
 
async def batch_generate_async(
    prompts: List[str],
    model_name: str = "gemini-2.5-flash",
    max_concurrent: int = 5
) -> List[str]:
    """
    Process a batch of prompts asynchronously.
    Semaphore prevents exceeding the concurrent request limit.
    """
    semaphore = asyncio.Semaphore(max_concurrent)
    model = genai.GenerativeModel(model_name)
 
    async def generate_one(prompt: str) -> str:
        async with semaphore:
            loop = asyncio.get_event_loop()
            response = await loop.run_in_executor(None, model.generate_content, prompt)
            return response.text
 
    results = await asyncio.gather(*[generate_one(p) for p in prompts], return_exceptions=True)
    return [r if not isinstance(r, Exception) else "" for r in results]

Production-Grade Client: Putting It All Together

import logging
import time
 
logger = logging.getLogger(__name__)
 
class ProductionGeminiClient:
    """
    Production-ready Gemini API client.
 
    Combines: adaptive rate limiting, multi-key pooling,
    exponential backoff, Cloud Monitoring metrics, and structured logging.
    """
 
    def __init__(
        self,
        api_keys: List[str],
        project_id: str,
        rpm_per_key: int = 60,
        max_retries: int = 5
    ):
        self.pool = GeminiApiKeyPool(api_keys, rpm_per_key)
        self.metrics = GeminiApiMetricsCollector(project_id)
        self.max_retries = max_retries
 
    def generate(self, prompt: str, model_name: str | None = None, **kwargs) -> str:
        if model_name is None:
            model_name = select_model(prompt)
 
        start_time = time.time()
        last_error = None
 
        for attempt in range(self.max_retries):
            entry = self.pool.get_healthy_key()
 
            try:
                genai.configure(api_key=entry.key)
                wait = entry.limiter.wait_if_needed()
                if wait > 0.5:
                    logger.info(f"Throttled for {wait:.2f}s")
 
                model = genai.GenerativeModel(model_name, **kwargs)
                response = model.generate_content(prompt)
 
                latency_ms = (time.time() - start_time) * 1000
                tokens = response.usage_metadata.total_token_count
 
                self.pool.record_key_success(entry, tokens)
                self.metrics.record_request(model_name, True, tokens, latency_ms)
                logger.info(f"✅ {model_name} | {tokens} tokens | {latency_ms:.0f}ms")
                return response.text
 
            except ResourceExhausted as e:
                last_error = e
                self.pool.record_key_error(entry)
                self.metrics.record_request(model_name, False, 0, 0)
 
                error_type = classify_rate_limit_error(e)
                sleep = {
                    "RPM_EXCEEDED": random.uniform(2 ** attempt, 2 ** (attempt + 1)),
                    "TPM_EXCEEDED": random.uniform(10, 30),
                }.get(error_type, random.uniform(1, 5))
 
                logger.warning(f"⚠️ {error_type} | attempt {attempt + 1}/{self.max_retries} | sleeping {sleep:.1f}s")
                time.sleep(sleep)
 
        raise last_error or RuntimeError("All retries exhausted")
 
# Initialize
client = ProductionGeminiClient(
    api_keys=[os.environ["GEMINI_API_KEY_1"], os.environ["GEMINI_API_KEY_2"]],
    project_id="your-gcp-project-id",
    rpm_per_key=60,
)
 
result = client.generate("Explain production API design patterns for high-traffic services.")
print(result)

Seven Implementation Insights You Won't Find in the Docs

These are things the official Vertex AI / Gemini API documentation does not call out, but that I have absorbed the hard way while running the wallpaper app backend. I keep them as a personal checklist for design reviews.

1. `usage_metadata` lags actual usage by 1–2 seconds

If you stream token usage straight from usage_metadata into Cloud Monitoring, your dashboard will under-report consumption for the first second or two of every spike. In my logs the gap between receiving a response and seeing it reflected on the Cloud Monitoring graph was 1.2 to 2.1 seconds on average.

# Recommended: track locally at send-time too
class LocalUsageTracker:
    def __init__(self):
        self.window = []  # (timestamp, tokens)
 
    def record(self, tokens: int):
        now = time.time()
        # 60-second sliding window
        self.window = [(t, n) for t, n in self.window if now - t < 60]
        self.window.append((now, tokens))
 
    def tpm(self) -> int:
        return sum(n for _, n in self.window)

I keep both: the local tracker for alerting, usage_metadata for billing audit.

2. The `Retry-After` header from Google APIs is not always present

google.api_core.exceptions.ResourceExhausted sometimes carries retry_after, sometimes not. Code defensively for both from day one.

def get_retry_delay(exc, attempt: int) -> float:
    hdr = getattr(exc, "retry_after", None)
    if isinstance(hdr, (int, float)) and hdr > 0:
        return min(float(hdr), 60.0)
    # Fallback: exponential backoff + jitter
    base = min(2 ** attempt, 32)
    return base + random.uniform(0, base * 0.25)

3. For indie developers, multi-key pooling means Vertex AI service accounts

Bundling personal API keys from multiple Google accounts is a grey-zone move under current ToS. For the wallpaper app I run three Vertex AI service accounts split by purpose (Flash, Pro, nightly batch), each with its own project-level quota. That gives you a legitimate multi-key pool with predictable accounting.

4. `ResourceExhausted` covers both 429 and TPM exhaustion

The exception class doesn't disambiguate. Inspect the message:

def classify_resource_exhausted(exc) -> str:
    msg = str(exc).lower()
    if "token" in msg or "tpm" in msg:
        return "tpm"
    if "request" in msg or "rpm" in msg:
        return "rpm"
    return "unknown"

If it's a TPM trip, short retries are pointless — the rolling window has to drain. I widen the backoff to 30–60 seconds or push the job back onto a queue.

5. Set timeouts per model, not per client

I split timeouts explicitly between Flash and Pro. Sharing a single timeout means Pro requests that would have returned get cut off and retried — double the cost, same answer.

Model	Timeout	Reason
`gemini-2.5-flash`	15–30 s	Latency-first, long responses rarely happen
`gemini-2.5-pro`	60–120 s	Reasoning-heavy tasks mix in
`gemini-2.5-pro` (high thinking)	120–180 s	Thinking tokens eat real wall time

6. Buffer Cloud Monitoring custom metrics for ~30 seconds

Calling metric_descriptor.write once per request runs up the write-count line on your bill. Buffering for 30 seconds and flushing in batches cut my Cloud Monitoring spend by roughly 60% month over month.

class BufferedMetricWriter:
    def __init__(self, flush_interval: float = 30.0):
        self.buffer: list[tuple[str, float, dict]] = []
        self.last_flush = time.time()
        self.flush_interval = flush_interval
 
    def record(self, name: str, value: float, labels: dict):
        self.buffer.append((name, value, labels))
        if time.time() - self.last_flush > self.flush_interval:
            self.flush()
 
    def flush(self):
        # batch-write to Cloud Monitoring here
        ...
        self.buffer.clear()
        self.last_flush = time.time()

7. Alert on success rate, not just threshold breaches

Even with no 429s in sight, you can have a silent outage if safety_ratings start landing responses on empty. An SLO alert like "success rate below 95% over the last 5 minutes" catches that class of failure. I route mine to Slack via webhook so I can react in place.

Measured Numbers from the Wallpaper App

Measurement	Value	Window
Avg tokens per request, nightly metadata batch	~3,200 tokens	2026-02 – 04
Effective RPM at which `gemini-2.5-pro` starts returning 429	around 950 RPM	same
429 recovery rate with backoff (base 1 s, max 32 s, jitter)	97% within 6 retries	same
Effective throughput gain from a 3-key pool	~2.4x vs single key	same
`usage_metadata` reporting lag	1.2 – 2.1 s	same
Cost reduction from 30-second metric buffering	~60%	2026-03 – 04

These are wallpaper-app workload numbers, so chat or long-form tasks will look different. But the shape — "Pro starts bleeding around 950 of its nominal 1,000 RPM" and "six retries reclaim ~97% of failed calls" — is a useful anchor when sizing capacity for the first time.

Decision Lines When Gemini Sits on Top of AdMob

In an app where AdMob ad revenue is the constant, Gemini API cost is directly a tax on that revenue. The simple decision rules I use:

Model selection

Real-time, user-triggered generation → gemini-2.5-flash, no exceptions. Pro is too slow and too expensive at this point in the funnel.
Nightly batches and thumbnail captions → Pro is fine, but cap RPM at monthly_budget / 30 / 1.3. The / 1.3 is a safety margin for spiky ad-hoc batches.
Design review and code analysis → Pro with thinking. Low RPM is acceptable.

Cost alerts

I set a daily Cloud Billing alert at monthly_budget / 30 * 1.3. That tolerates a 33% daily overshoot before paging me. AdMob is monthly settlement, so blowing the budget mid-month means the app's core features (pre-roll thumbnail generation, for example) start to throttle and ad revenue drops with them.

Key separation

I split the Vertex AI service accounts three ways:

Flash key (user-facing) — failure here hurts most. Single key with maximum headroom rather than a pool.
Pro key (heavy analysis / batches) — failures here are mostly invisible to users.
Reserve / ad-hoc batch key — insurance for when 1 and 2 are both pinned.

That redundancy is what lets me not check my phone overnight, which is genuinely the biggest practical win.

Per-Task RPM Allocation Cheatsheet

Sizing each task's RPM budget up front prevents most of the surprise 429 incidents. My current numbers:

Task	Model	Target RPM	Notes
Chat-style response to user input	Flash	60–120	Latency-first
Single-image captioning	Flash	30–60	TPM trips before RPM
Article summarisation batch	Flash	200–400	Adaptive + multi-key
Wallpaper metadata regeneration (nightly)	Pro	50–80	Cost-first, throttle aggressively
Design review / code analysis	Pro	10–20	Heavy per-request payload
Classification with reasoning	Pro + thinking	5–15	Timeout ≥ 120 s

I deliberately leave the Flash 200–400 RPM band unused at steady state so I can spike a batch without paging anyone.

A Few Practical Questions

Q. Can indie developers just bundle multiple personal Google accounts' keys?

A. There's real ToS risk. Use Vertex AI service accounts split by purpose — that's what I've migrated to.

Q. Should retry intervals for 429 and 503 differ?

A. They should. 429 recovers fine with short backoff (start at 1 s, exp + jitter), but 503 is a server-side load issue and benefits from 10–30 second initial waits. My ProductionGeminiClient catches both with (ResourceExhausted, ServiceUnavailable) and branches internally.

Appendix: tenacity and Node.js Variants

The examples so far hand-roll exponential backoff in Python. In practice two needs show up: not rewriting the same wait logic for every function, and calling the API from a non-Python runtime with the same policy. Here are the two variants I actually reach for.

Declarative Retries with tenacity

When the retry condition and the wait ceiling are fixed, declaring them with tenacity as a decorator keeps the core logic readable. I find it safer to attach @retry to a narrow target than to wrap every call in a hand-written call_with_backoff.

from tenacity import (
    retry,
    stop_after_attempt,
    wait_exponential,
    retry_if_exception_type,
)
from google.api_core.exceptions import ResourceExhausted
 
@retry(
    retry=retry_if_exception_type(ResourceExhausted),
    wait=wait_exponential(multiplier=1, min=2, max=60),
    stop=stop_after_attempt(6),
)
def generate_with_retry(prompt: str) -> str:
    response = model.generate_content(prompt)
    return response.text
 
# Calls automatically retry on rate limit errors
text = generate_with_retry("Give me an overview of machine learning")

Capping the wait at 60 seconds with wait_exponential(min=2, max=60) keeps an overnight batch from stalling indefinitely, and retry_if_exception_type(ResourceExhausted) makes sure you only swallow rate-limit errors rather than silently absorbing exceptions you actually want to see.

Calling from Node.js / TypeScript

If your admin tooling or front-facing code is in TypeScript, the same "max concurrency + minimum interval" idea ports directly. You don't need anything as elaborate as the Python AdaptiveRateLimiter; a minimal queue alone cuts 429s noticeably.

import { GoogleGenAI } from "@google/genai";
 
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY! });
 
class RateLimiter {
  private queue: Array<() => void> = [];
  private running = 0;
 
  constructor(private maxConcurrent: number, private delayMs: number) {}
 
  async execute<T>(fn: () => Promise<T>): Promise<T> {
    while (this.running >= this.maxConcurrent) {
      await new Promise<void>((resolve) => this.queue.push(resolve));
    }
    this.running++;
    try {
      const result = await fn();
      await new Promise((r) => setTimeout(r, this.delayMs));
      return result;
    } finally {
      this.running--;
      this.queue.shift()?.();
    }
  }
}
 
// RPM=60 → 1-second intervals, max 5 concurrent
const limiter = new RateLimiter(5, 1000);
 
async function safeGenerate(prompt: string): Promise<string> {
  return limiter.execute(async () => {
    const response = await ai.models.generateContent({
      model: "gemini-2.5-flash",
      contents: prompt,
    });
    return response.text ?? "";
  });
}

Tune maxConcurrent and delayMs to your RPM and the front end and back end can share one rate policy. I keep the backend on the Python production client and use this lighter version only for auxiliary Node.js tooling.

Closing Thoughts

Rate limiting is the kind of system that earns its keep quietly. It does not produce flashy bugs, but a single bad day can shift your work plan by 24 to 48 hours. Rather than waiting for the incident, the four cheapest things to put in before your first production cutover are: exponential backoff with jitter at the request level, per-model timeouts, key separation by purpose, and a daily cost alert.

If you want one concrete next step from this article, check whether the Gemini client you currently run uses the same timeout for Flash and Pro. Splitting just that one value tends to cut both surprise overnight pages and the slow drip of double-charged retries.

Thanks for reading — if any of these notes save you a bad night, that's exactly what I was hoping for.

Thank You for Reading

Gemini Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

✦Copy-paste ready implementation code
✦New advanced guides published daily
✦$5/mo or $10 for lifetime access

View Membership →