Async AI Job Queues with Gemini API and Cloud Tasks — Production Patterns for Timeouts, Retries, and Rate Limits

Sizing the queue against Gemini's quota

Assume you are on a plan that gives you roughly 2 RPS / 60 RPM on Gemini 2.5 Pro. Stay conservatively below that and reserve headroom for ad-hoc calls. Check the exact numbers in the Google AI Studio quota page — they vary by plan and region, and Google occasionally bumps them in response to general availability announcements.

# Create the ai-jobs queue
gcloud tasks queues create ai-jobs \
  --location=asia-northeast1 \
  --max-dispatches-per-second=1.5 \
  --max-concurrent-dispatches=4 \
  --max-attempts=5 \
  --min-backoff=10s \
  --max-backoff=600s \
  --max-doublings=4

What each knob means:

• --max-dispatches-per-second=1.5 — queue-side RPS ceiling. Setting it below Gemini's limit gives you headroom for manual/debug calls. Remember that your quota is per project, so if the same project powers a web app, a Slack bot, and a batch job, they all compete for that 2 RPS.
• --max-concurrent-dispatches=4 — how many worker invocations can be in flight simultaneously. Match this to your worker's --max-instances; otherwise Cloud Tasks will try to dispatch more work than Cloud Run is willing to run. The two numbers should move together when you tune throughput.
• --max-attempts=5 — five tries, then give up. Beware of 100 as a default — permanent errors will eat quota 100 times, and a single malformed prompt can cost serious money. Five is a reasonable starting point because transient failures rarely require more than three or four retries if your backoff is set correctly.
• --min-backoff=10s --max-backoff=600s --max-doublings=4 — exponential backoff (10 → 20 → 40 → 80 → 160s). Plenty of room for the worst bursts without pinning your queue in a permanent retry storm. If you see failures clustering in tight windows and then recovering, lengthen min-backoff; if recovery takes hours, the problem is not backoff — it is a permanent error that should not be retried.

The API layer — enqueue and return

Keep the API thin. FastAPI on Cloud Run is the example below; Node/Express is structurally identical.

# job_api.py — accept the request and enqueue a Cloud Tasks job
import os
import json
import uuid
from datetime import datetime, timezone
from fastapi import FastAPI, HTTPException
from google.cloud import firestore, tasks_v2
from pydantic import BaseModel
 
PROJECT = os.environ["GOOGLE_CLOUD_PROJECT"]
LOCATION = "asia-northeast1"
QUEUE = "ai-jobs"
WORKER_URL = os.environ["WORKER_URL"]
WORKER_SA = os.environ["WORKER_INVOKER_SA"]
 
app = FastAPI()
db = firestore.Client()
tasks = tasks_v2.CloudTasksClient()
QUEUE_PATH = tasks.queue_path(PROJECT, LOCATION, QUEUE)
 
class AnalyzeRequest(BaseModel):
    user_id: str
    input_text: str
 
@app.post("/jobs/analyze", status_code=202)
def enqueue(req: AnalyzeRequest):
    if len(req.input_text) > 1_000_000:
        raise HTTPException(413, "input too large")
 
    job_id = str(uuid.uuid4())
    now = datetime.now(timezone.utc).isoformat()
    db.collection("jobs").document(job_id).set({
        "userId": req.user_id,
        "status": "queued",
        "progress": 0,
        "createdAt": now,
        "updatedAt": now,
        "input": req.input_text[:2000],
    })
 
    task = {
        "http_request": {
            "http_method": tasks_v2.HttpMethod.POST,
            "url": WORKER_URL,
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps({"jobId": job_id}).encode(),
            "oidc_token": {"service_account_email": WORKER_SA, "audience": WORKER_URL},
        },
        "dispatch_deadline": {"seconds": 1800},  # give up after 30 min
    }
    tasks.create_task(parent=QUEUE_PATH, task=task)
    return {"jobId": job_id, "status": "queued"}

Expected behavior: POST returns in 100–300 ms, a job document exists in Firestore, Cloud Tasks fires the worker within seconds. The oidc_token field tells Cloud Run to only accept this invocation from a task dispatched with the correct service account — no public caller can hit your worker directly. If you drop that field and leave the worker's IAM allowing unauthenticated invocations, an attacker who discovers your worker URL can trigger Gemini calls on your dime. Locking the worker behind Cloud Run IAM plus oidc_token is the difference between a hobby script and a production deployment.

The dispatch_deadline of 1800 seconds caps how long Cloud Tasks will keep trying a single task before declaring it failed. Thirty minutes is generous enough for even heavy long-context Gemini jobs. If your worker is genuinely still running at the 30-minute mark, the architecture should probably stage the work (see the staged-pipeline section below) rather than lengthen the deadline further.

A word on state machines before we write code

Before jumping into the worker, it helps to draw the state machine on paper. Every job transitions through the same four states — queued, running, completed, failed — and one optional state, cancelled. The transitions that matter are the ones between queued and running, and from running into any terminal state. Everything else is either a guard or a no-op.

The rule I enforce everywhere is that terminal states are absorbing. Once a job is completed or failed, no subsequent invocation should change it. That single invariant eliminates most of the "mysterious duplicate charge" bugs I see in production systems. When you find yourself tempted to "just flip it back to running and retry", write a new jobId instead — the audit log is far cleaner.

A second useful discipline: every field you write to a job document should include updatedAt. Even if you never query by it, having a uniform timestamp on every mutation makes forensic investigation much easier the day something goes wrong. The five bytes of storage are absolutely worth it.

The worker — idempotent and re-entrant

Cloud Tasks is at-least-once: if a 2xx response is lost to a transient network blip, the task is redelivered. So your worker must be safe to run again. This is the single most important property of the whole design — if you skip idempotency, every transient failure is also a duplicate Gemini call and a duplicate result write. The state transitions below use Firestore transactions to guarantee that only one invocation ever flips queued → running and that completed/failed are terminal. A job that has already finished will never run again; a job that is currently running will not be taken over by a second worker.

# job_worker.py — the worker that talks to Gemini
import os
import json
from fastapi import FastAPI, HTTPException, Request
from google.cloud import firestore
from google import genai
from google.genai import types
 
PROJECT = os.environ["GOOGLE_CLOUD_PROJECT"]
GEMINI_API_KEY = os.environ["GEMINI_API_KEY"]
 
app = FastAPI()
db = firestore.Client()
client = genai.Client(api_key=GEMINI_API_KEY)
 
def _transition(job_ref, expected_from, to, extra=None):
    @firestore.transactional
    def txn(transaction):
        snap = job_ref.get(transaction=transaction)
        if not snap.exists:
            return False
        if snap.get("status") \!= expected_from:
            return False
        payload = {"status": to, "updatedAt": firestore.SERVER_TIMESTAMP}
        if extra:
            payload.update(extra)
        transaction.update(job_ref, payload)
        return True
    return txn(db.transaction())
 
@app.post("/run")
async def run_job(req: Request):
    body = await req.json()
    job_id = body.get("jobId")
    if not job_id:
        raise HTTPException(400, "jobId required")
 
    job_ref = db.collection("jobs").document(job_id)
    snap = job_ref.get()
    if not snap.exists:
        return {"status": "missing"}
 
    status = snap.get("status")
    if status in ("completed", "failed"):
        return {"status": status}  # idempotent: never re-run
 
    acquired = _transition(job_ref, "queued", "running", {"startedAt": firestore.SERVER_TIMESTAMP})
    if not acquired and status \!= "running":
        return {"status": "conflict"}
 
    try:
        input_text = snap.get("input")
        response = client.models.generate_content(
            model="gemini-2.5-pro",
            contents=input_text,
            config=types.GenerateContentConfig(
                system_instruction="Structure the input and return JSON.",
                response_mime_type="application/json",
                temperature=0.2,
            ),
        )
        _transition(job_ref, "running", "completed", {
            "result": response.text,
            "finishedAt": firestore.SERVER_TIMESTAMP,
            "progress": 100,
        })
        return {"status": "completed"}
 
    except Exception as e:
        code = getattr(e, "status_code", None) or getattr(e, "code", None)
        if code in (429, 500, 502, 503, 504):
            job_ref.update({"lastError": str(e)[:500], "updatedAt": firestore.SERVER_TIMESTAMP})
            raise HTTPException(503, "retryable")  # let Cloud Tasks retry
        _transition(job_ref, "running", "failed", {
            "error": str(e)[:500],
            "finishedAt": firestore.SERVER_TIMESTAMP,
        })
        return {"status": "failed"}

The worker code follows a simple, deliberate contract that you can describe in one sentence: a successful result is a 2xx, a transient failure is a 503, and a permanent failure is a 2xx with status=failed written to Firestore. Everything else flows from that rule. To restate it explicitly:

• First delivery: queued → running → completed, result stored in Firestore.
• Redelivery of the same jobId: hits the terminal-state guard and exits, so you are never billed twice.
• 429 / 5xx from Gemini: return HTTPException(503) and let Cloud Tasks retry with backoff.
• Permanent errors: mark failed and return 2xx so the queue stops retrying.

Never retry inside the worker and inside the queue. Doubling up turns 5 × 5 into 25 Gemini calls on one logical job. Pick one layer to own retries — the queue in our case — and strip retry logic from the other. Writing try/except blocks that swallow exceptions and call Gemini again feels defensive, but it quietly undermines your quota accounting and makes incidents far harder to debug.

When the worker should not trust itself

One nuance that trips up many implementations: the worker must assume that another copy of itself may also be running for the same jobId. The idempotency guard prevents damage, but it does not prevent two invocations from both calling Gemini before the guard fires. If you write results in streaming mode, where each chunk updates Firestore as it arrives, you can observe this directly: two runs produce interleaved writes that corrupt the final output.

The safe pattern is to accumulate the result in the worker's local memory and write it to Firestore only once, in the final _transition from running to completed. If the transition fails because another worker beat you to it, your local result is discarded without harm. Users may occasionally see a brief "running" state flicker, but the final result is always consistent.

This matters more than it first appears. In my logs, roughly one in a thousand tasks has a genuine redelivery where the first attempt's 2xx was lost in flight. Without the "write once, at the end" pattern, every one of those would produce a corrupted job document.

Progress updates through Firestore snapshots

Users will abandon a silent progress bar. For anything that takes longer than ~10 seconds, write incremental progress to the job document and let the client subscribe directly. The subjective difference between "a spinner for 90 seconds" and "a progress bar creeping from 10% to 95%" is enormous, and it costs you almost nothing — one extra Firestore write per stage.

def update_progress(job_ref, p: int, note: str):
    job_ref.update({
        "progress": p,
        "note": note,
        "updatedAt": firestore.SERVER_TIMESTAMP,
    })
 
for i, image in enumerate(images):
    analyze_one(image)
    update_progress(job_ref, int((i + 1) / len(images) * 90), f"{i+1}/{len(images)} done")
update_progress(job_ref, 100, "completed")

// client.js
import { getFirestore, doc, onSnapshot } from "firebase/firestore";
const db = getFirestore();
 
function subscribeJob(jobId, onUpdate) {
  return onSnapshot(doc(db, "jobs", jobId), (snap) => {
    if (\!snap.exists()) return;
    const data = snap.data();
    onUpdate({
      status: data.status,
      progress: data.progress || 0,
      note: data.note || "",
      result: data.result,
    });
  });
}

onSnapshot handles reconnects automatically and sends diffs only, which is cheaper than polling and cleaner than long-lived WebSockets. Users who leave the page and come back see the correct current state without another round trip to your backend, because the Firestore SDK rehydrates from cache and then catches up.

Metric sampling and how it affects alert fidelity

Cloud Monitoring samples custom metrics at most once per minute by default, which can hide fast spikes. For latency work, either write to a distribution metric or consider buffering samples in a background task and flushing as a gauge on a 10-second cadence. The latter is more code but catches the bursty behavior that matters most for Gemini traffic.

If you only remember one thing about sampling: your dashboard cannot show you anything finer than its sample rate, and Gemini's slowest responses are often brief outliers inside an otherwise healthy minute. If your P99 looks unreasonably close to your P50, you probably have a sampling issue, not a performance one.

Staged pipelines — re-enqueue instead of running a 20-minute worker

Real pipelines rarely fit in one call: "parse PDF → summarize each chapter → integrate → proofread". Running that in one worker bumps against dispatch deadlines. Split each stage into its own task and have the worker enqueue the next stage before returning 2xx.

def enqueue_next(job_id: str, stage: str):
    task = {
        "http_request": {
            "http_method": tasks_v2.HttpMethod.POST,
            "url": f"{WORKER_URL}?stage={stage}",
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps({"jobId": job_id}).encode(),
            "oidc_token": {"service_account_email": WORKER_SA, "audience": WORKER_URL},
        },
        "dispatch_deadline": {"seconds": 900},
    }
    tasks.create_task(parent=QUEUE_PATH, task=task)
 
if stage == "chapter_summary":
    # ... Gemini call ...
    job_ref.update({"progress": 60, "stage": "chapter_summary_done"})
    enqueue_next(job_id, "integrate")
    return {"status": "chapter_summary done"}

A second benefit: each stage can live on a different queue with different RPS settings. The summary stage might run at concurrency 8; the integrate stage at concurrency 1. You cannot do that in a monolithic worker.

When progress writes become the bottleneck

When Gemini streams hundreds of tokens per second and you want to mirror that into Firestore for a visible word-by-word UI, the write rate becomes the scarce resource. Firestore charges per write, and at 10 writes per second per document you quickly hit both cost and quota walls. I use two tactics to keep progress snappy without going broke:

First, throttle progress writes to one per 500 ms. The human eye cannot distinguish finer than that anyway. A tiny debounce inside the worker loop is enough.

Second, accumulate the partial text in memory and flush the running buffer only every N tokens. Users see a roughly smooth stream, your Firestore bill stays flat. This pattern turns what would be a thousand-write-per-job nightmare into a ten-write-per-job job.

Cancellable jobs

You cannot interrupt a running Cloud Tasks invocation, but you can make the worker check a cancelRequested flag at stage boundaries.

def is_cancelled(job_ref) -> bool:
    snap = job_ref.get()
    return bool(snap.exists and snap.get("cancelRequested"))
 
if is_cancelled(job_ref):
    _transition(job_ref, "running", "cancelled", {"finishedAt": firestore.SERVER_TIMESTAMP})
    return {"status": "cancelled"}

If you use streaming Gemini calls, check is_cancelled inside the chunk loop — but no more than once every 5 seconds so you do not explode your Firestore read count.

Why cancellation must be cooperative

Cloud Tasks offers no way to forcibly stop a running invocation. This sounds like a limitation until you realize it is actually a feature: if kills were involuntary, Gemini could still produce side effects (logging, billing, partial writes) during the kill window. Cooperative cancellation — where the worker checks a flag and exits cleanly — gives you deterministic behavior at every cancellation point.

Pick your cancellation checkpoints carefully. Between stages is always safe. Inside a single long Gemini call, you can only cancel at stream boundaries. For short calls, it is usually fine to let them finish and discard the result. Write this decision down per endpoint so that when a user reports "I hit cancel and it took 30 seconds to stop", you can explain exactly which checkpoint they were waiting for.

Dead letter queue — where failures become improvements

By default Cloud Tasks discards a task after max-attempts. In production that is unacceptable. Capture failures with a tiny check in the worker:

attempt = int(req.headers.get("X-CloudTasks-TaskRetryCount", "0"))
if attempt >= 4:  # last try before the queue gives up
    db.collection("dead_letters").document(job_id).set({
        "jobId": job_id,
        "lastError": str(e)[:1000],
        "attempts": attempt + 1,
        "movedAt": firestore.SERVER_TIMESTAMP,
    })

After a few weeks of traffic, DLQ entries usually cluster into three buckets, and I strongly recommend classifying them as they arrive — the taxonomy accelerates triage tenfold compared to staring at raw stack traces:

• Oversize input — a document with over 1M tokens slipped through. Pre-validate at the API layer.
• Safety filter blocks — user-generated inputs often hit prompt_feedback blocks. Log response.prompt_feedback.block_reason.
• Model drift — a minor Gemini release subtly changed response formatting. Pin the model version in production code.

Error taxonomy your DLQ writer should know

The three buckets above are the common case, but in practice there is a fourth: intermittent model unavailability. Every few months Gemini has a regional blip where 503s spike for ten or fifteen minutes. Tasks dispatched during that window will usually succeed on their next retry, but some will hit max-attempts before the outage clears. Tagging these in your DLQ with a distinct outage_suspected label makes it easy to bulk-requeue them after the fact. I keep a small script that reads the DLQ, filters on a time window, and reposts to the queue — it has saved me more than once.

Observability with Cloud Monitoring

Make latency and token usage visible, not just cost:

import time
start = time.time()
response = client.models.generate_content(...)
elapsed_ms = int((time.time() - start) * 1000)
job_ref.update({
    "geminiLatencyMs": elapsed_ms,
    "inputTokens": response.usage_metadata.prompt_token_count,
    "outputTokens": response.usage_metadata.candidates_token_count,
})

Three custom metrics are enough to start: gemini/latency_ms, gemini/input_tokens, gemini/output_tokens. Set one alert on P95 latency above 30 seconds and one on daily token totals above 200% of yesterday. Resist adding more — unused alerts train teams to ignore alerts, and the cost of a missed real incident is far higher than the cost of a slightly sparse dashboard. You can always add metrics later once you have seen what your traffic actually does.

Dashboards worth keeping:

• P50 / P95 latency per stage.
• Failure ratio (status=failed / total).
• DLQ entries per day.
• Queue depth via cloudtasks.googleapis.com/queue/depth.

Designing for graceful degradation

A queue-based design makes graceful degradation almost free. If Gemini starts returning 429s at a higher rate than normal, drop max-dispatches-per-second by half for the duration of the incident. Jobs still land, they just process slower. Users see the progress bar move — which is a far better failure mode than seeing 504 errors and abandoning.

For scheduled batch work, add a cheap "deprioritize" flag to the job document and skip deprioritized jobs when queue depth is above a threshold. I have used this to protect paying users' interactive requests during overnight batch imports.

Cost math versus a synchronous deployment

At 5,000 jobs/day, avg 45s per Gemini call, concurrency 8:

• Sync (Cloud Run gen-2, min-instances≥2): idle CPU and blocked response slots push you to $120–$150/month of compute. Failure rate 5–8%.
• Async (API min=0, Worker min=0, maxInst=8): API returns in <300 ms, Worker bills only while Gemini is running. Same workload lands at $40–$55/month with <1% failure.

Savings scale with burstiness. Flat traffic gets less benefit; spiky workloads gain the most. Measure the ratio between your busiest hour and your quietest hour before committing to the migration — if the ratio is 3× or more, async is almost certainly worth the refactor cost.

Distinguishing transient from permanent in your logs

Once you have latency and token metrics, the next most valuable panel is a stacked bar chart of outcomes by type: completed, failed_permanent, failed_transient_exhausted. The first category is your happy path. The second tells you about bugs and prompt quality. The third tells you whether your retry settings are right — if it grows, Gemini is genuinely flaky or your concurrency is too aggressive.

I tag every Firestore write with a small outcomeClass field so queries against this are O(1). The extra write is a few microseconds; the reduction in debugging time is hours per incident.

Local development without a Cloud Tasks emulator

There is no Cloud Tasks emulator, but a thin "invoke the worker directly" shim gives you near-prod behavior offline.

import os, requests
 
LOCAL_MODE = os.environ.get("LOCAL_MODE") == "1"
 
def enqueue_or_invoke(url, body, sa):
    if LOCAL_MODE:
        r = requests.post(url, json=body, timeout=600)
        r.raise_for_status()
        return {"status": "invoked_local"}
    # ... production Cloud Tasks call ...

Combined with the Firestore emulator, this lets you run four critical regression tests:

• The same jobId invoked twice calls Gemini exactly once.
• A job with cancelRequested=true exits cleanly mid-stage.
• A simulated 429 returns 503 so the queue retries.
• An oversize input lands in failed and never retries.

I can write these four tests in pytest + httpx in under 30 minutes. The return on that half hour is catching most production bugs before they ship.

Shipping the worker with CI that knows about staged rollouts

Before security, a note on deployment hygiene. Cloud Run supports traffic splitting by revision, and the worker is exactly the kind of service where you want that. Deploy with --no-traffic, send 1% through the new revision for a few hours, watch the DLQ and P95 latency panels, then shift to 100%. Because Cloud Tasks reads WORKER_URL from your environment, the same task queue will fan traffic across both revisions naturally during the rollout window.

If a regression slips through, gcloud run services update-traffic rolls back to the previous revision in seconds. The combination of declarative queue settings and revisioned Cloud Run gives you incident response primitives that are hard to match in self-hosted systems.

Security and the principle of least privilege

Build these three controls from day one, because retrofitting them is painful:

• Separate service accounts for job-api and job-worker. Only the worker should see the Gemini API key. If the API layer is ever compromised, the key still is not.
• Secret Manager injection for GEMINI_API_KEY, never a plain env var. Rotation becomes a one-line operation.
• Strict Firestore rules so clients can only read their own jobs. Writes are disallowed from clients entirely — the worker writes through the Admin SDK.

// firestore.rules
match /jobs/{jobId} {
  allow read: if request.auth \!= null && resource.data.userId == request.auth.uid;
  allow write: if false;
}

Trade-offs you inherit

Async is not free. Before adopting it, agree with yourself that you accept:

• Two Cloud Run services to manage, two deploy pipelines, two log streams.
• Clients must handle a two-phase pattern (enqueue → subscribe) instead of a single response.
• Firestore read counts go up (though Cloud Run compute goes down more).
• Debug paths lengthen: API → Tasks → Worker → Firestore → Client. Invest in a correlation id threaded through every log line.

My heuristic: if your Gemini call reliably completes under 15 seconds, stay synchronous. Migration pays off past that threshold, and the operational complexity stops being worth it if every call is fast. Think of async as insurance against the long tail, not a default pattern to apply everywhere.

Committing the queue settings to Cloud Tasks

Treat the queue definition as code, not as a one-off gcloud call. A small Terraform block keeps the settings reviewable.

resource "google_cloud_tasks_queue" "ai_jobs" {
  name     = "ai-jobs"
  location = "asia-northeast1"
 
  rate_limits {
    max_dispatches_per_second = 1.5
    max_concurrent_dispatches = 4
  }
 
  retry_config {
    max_attempts       = 5
    min_backoff        = "10s"
    max_backoff        = "600s"
    max_doublings      = 4
  }
}

Now "we raised concurrency to 8 on 2026-04-22" is a git diff, not a ticket.

Documenting the contract the pipeline enforces

One underappreciated benefit of the async pattern is that it creates a natural boundary to document. Write down exactly what /jobs/analyze does, what schema the client can expect in the Firestore subscription, and what states a job can end in. Anyone onboarding to the team can read that page and know what each layer is responsible for.

I have seen projects collapse into confusion over time because this one page did not exist. The API team assumed the worker guaranteed retries; the worker team assumed the API pre-validated input; neither was fully right. A single README in the repo root closes the gap.

What to recheck one month after launch

A month in, review these five knobs once and your next six months of operations will be much calmer:

• Is max-attempts=5 still right? If most successes happen on attempt 1–2, drop it to 3.
• Is min-backoff longer than your P50 worker duration? Shorter settings trigger idempotency-guarded redeliveries that burn budget for nothing.
• Can you raise max-concurrent-dispatches? If your Gemini quota has headroom, throughput is free.
• Top three DLQ buckets: fold pre-validation for them into the API layer.
• Break down unit economics: what does one job cost across Cloud Run + Cloud Tasks + Firestore + Gemini? If it is 0.3 yen you can be generous; at 3 yen you need guardrails.

Stick this review on your calendar the day you deploy, so it is not forgotten.

Common questions at design reviews

Why not Pub/Sub?
Pub/Sub is excellent for fan-out, but it does not cap dispatch rate out of the box. Gemini's quota is RPS-shaped, so Cloud Tasks' max-dispatches-per-second maps directly onto it. For one-at-a-time AI work, Cloud Tasks is one configuration step simpler.

Why not Google Cloud Workflows?
If your pipeline is a fixed DAG with a handful of steps, Workflows is a clean choice. When routing depends on the output of a previous Gemini call, Cloud Tasks plus a worker gives you the flexibility of plain code. I use Workflows for ≤5 static stages, Cloud Tasks for dynamic pipelines.

Treat the pipeline itself as a product you maintain. The code is more durable than most features, and the operational leverage you get from a clean queue design is hard to overstate. Every time a teammate asks "should this endpoint be sync or async?" you now have a one-line answer backed by data. The day you stop treating queues as an exotic infrastructure choice and start treating them as the default shape for anything touching a slow API is the day Gemini integrations get genuinely boring to run.

Further reading

Next step

Pick one existing Cloud Run handler that calls Gemini and split just that one into an async pair. You do not need to migrate the whole system at once — the first migration teaches you everything the remaining ones need, and the operational relief from removing a single flaky endpoint is usually enough motivation to finish the rest. Start with the slowest or most-complained-about endpoint, because its improvement will be the most visible and the easiest to justify the work against.

For complementary context, see Gemini API × Cloud Run serverless production guide and Gemini API production error handling & rate limit guide.