Classifying 8,000 App Reviews Overnight with Gemini Batch API — and Moving Polling to Webhooks

Numbers Side-by-Side

The same 8,000 reviews, once through the regular API across a full day, and once through the Batch API overnight (gemini-2.5-flash, Tokyo region, my own environment as of May 2026):

Cost dropped to roughly half and runtime shrank by about 40%. The part I appreciated most as a solo developer was the zero-failure column. Not having to wake up to babysit the rate-limit recovery was a bigger psychological win than the cost savings. When you are the only person running six apps, that kind of quiet matters.

Also worth noting: the regular API number does not include the duplicate token cost from 429 retries (those still bill). The Batch API is much easier to forecast for that reason alone.

Stop Polling — Receive the Completion Instead, with Webhooks

This is the part I added in response to the June 2026 changes.

"Kick it off and sleep" was solved, but to read the result I still needed a morning cron (or a manual check) to call client.batches.get(...). Whether the job finished at 3am or 9am, the output sat in limbo until my script went to look.

In June 2026 the Gemini API gained event-driven Webhooks that replace polling for the Batch API and long-running operations. The idea is simple: instead of me pulling the state, Google pushes a notification to my endpoint the moment the job finishes. On receiving the completion, I let the download and write-back run in the same flow.

The receiver is just a thin HTTP endpoint with signature verification. I bolted it onto an existing notification FastAPI service.

import hashlib
import hmac
import os
from fastapi import FastAPI, Request, HTTPException
from google import genai
 
app = FastAPI()
client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])
WEBHOOK_SECRET = os.environ["GEMINI_WEBHOOK_SECRET"].encode()
 
def verify_signature(body: bytes, signature: str) -> bool:
    expected = hmac.new(WEBHOOK_SECRET, body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, signature)
 
@app.post("/gemini/batch-callback")
async def batch_callback(request: Request):
    raw = await request.body()
    sig = request.headers.get("X-Goog-Signature", "")
    if not verify_signature(raw, sig):
        raise HTTPException(status_code=401, detail="bad signature")
 
    event = await request.json()
    # Only act on completion; ack everything else (e.g. pending updates) with 200
    if event.get("state") != "JOB_STATE_SUCCEEDED":
        return {"ok": True, "skipped": event.get("state")}
 
    job_name = event["name"]  # batches/xxxx
    if already_processed(job_name):      # idempotency guard (below)
        return {"ok": True, "dedup": True}
 
    job = client.batches.get(name=job_name)
    result_blob = client.files.download(file=job.dest.file_name)
    write_back_to_firestore(result_blob)
    mark_processed(job_name)
    return {"ok": True}

When you create the job, you pass this receiver URL as the notification target. Field names are likely to shift right after launch, so put the notification config under config and reconcile the exact keys against the changelog.

batch_job = client.batches.create(
    model="gemini-2.5-flash",
    src=uploaded.name,
    config={
        "display_name": "review-classification-overnight",
        # Notification config added in 2026-06; verify keys as the API is new
        "notification": {
            "webhook_url": "https://api.example.com/gemini/batch-callback",
        },
    },
)

This is where the composite JSONL key pays off. Webhooks are designed to be redelivered, so the same completion event can arrive twice. Beyond the already_processed(job_name) idempotency guard on the receiver, a stable key for merging result lines back onto the original review means a duplicate notification produces the same write-back. Moving from "pull" to "receive" changes the assumptions about redelivery and ordering, so idempotency should be built in from the start rather than bolted on.

In my setup this let me delete an entire morning cron. Even when a job finishes in the middle of the night, by the time I wake up the classified results are already in Firestore and only the bug + severity=high notification has reached me. With no process sitting around waiting, I could also drop Cloud Run's minimum instances back to zero.

When You Should Keep Polling

Full migration to Webhooks is not always the right call. Here is the line I draw, running both:

• In environments where you cannot expose a receiver endpoint (local-only, inside a corporate network), keep the batches.get polling.
• As insurance against a dropped notification, keep a separate once-a-day reconcile that calls batches.get on any job still recorded as incomplete.
• Weekly, cross-check your notification receive logs against the actual JOB_STATE to confirm nothing slipped through.

Webhooks are fast and quiet, but their weakness is that you cannot notice a notification that never arrived. Even on a receive-based design, keeping one thin polling pass as a final safety net raises the reliability of the whole overnight pipeline a notch.

Three Things I Wish I Had Known

The implementation itself was small. Operating it taught me three things.

First, make the key truly unique. I started by using the raw Firestore document ID as the JSONL key. For one of my apps a few rows had the same review ingested twice in the past, and the duplicate key values caused result lines to overwrite each other — a small number of reviews silently disappeared from the merged output. A composite key like {app_id}-{review_id} removes the risk, and as noted above it also makes you resilient to Webhook redelivery.

Second, know when to give up on a hung job. The Gemini Batch SLA is 24 hours, but I have seen jobs stay in JOB_STATE_PENDING for over 12 hours without movement. Cancelling and resubmitting is usually faster than waiting. client.batches.cancel(name=...) is the call. Cancelling on Gemini costs almost nothing, so the decision is light.

Third, don't trust responseSchema blindly even at temperature: 0.1. On rare occasions summary_en came back as an empty string, even though the schema marked it required. Before writing back to Firestore I now always check if not result["summary_en"]: continue. The downstream daily digest email that uses these summaries would otherwise ship with a blank subject line.

Put Dated Deprecations on the Operations Calendar

One more thing that running multiple apps solo drives home: the importance of not missing dated deprecations.

In June 2026, for example, the image preview models gemini-3.1-flash-image-preview and gemini-3-pro-image-preview were announced for shutdown on 6/25. If, like me, you have a preview model wired into OGP image or wallpaper prototyping, your overnight pipeline quietly fails with a 404 the moment the cutoff passes. A receive-based design with Webhooks does nothing for you if the model you are submitting to no longer exists.

My countermeasure was to consolidate every referenced model ID into a single config file and transcribe the shutdown dates from ai.google.dev/gemini-api/docs/deprecations onto a calendar by hand. It is unglamorous, but when several overnight jobs share the same model constant, the swap is a one-line change. The "receive it fast" mechanism and the "never miss a deadline" discipline work as a pair in automated operations.

What I'm Trying Next

Looking at the 8,000 classified results, bug + severity=high + needs_reply=true came out to about 200 rows. I am building a follow-up pipeline that pushes those into gemini-2.5-pro to draft reply candidates. The Batch API is slow when you push long detailed prompts through it, so I am still undecided on whether to keep the reply-drafting step on Batch or move it back to the regular API. I will measure both before committing.

Running six apps alone means that introducing a tool like the Gemini Batch API, and spending the freed time on the reviews that genuinely deserve a careful reply, leads to better answers for users in the end. Layering Webhooks on top to "receive the completion" lightened the attention the overnight job demands by another notch. If you are at a similar scale and facing a similar wall, I hope this is useful.