A twice-daily batch that only ran once — reconstructing run counts from artifacts
One half of a scheduled job silently never fired, and throughput sat at half of plan for over a week without a single error in the logs. Here is how I reconstructed actual run counts from artifacts and backlog, with working code.
Retrying a Retired Model Never Showed Up in Success Latency
With the image generation models shutting down soon, I rebuilt a mock server to see what my retry layer actually does when it hits a retired model ID. The damage landed in wall time and queue wait, and never touched the metric I was watching.
When Version Numbers Stopped Meaning Generations: Rebuilding Cost Attribution That Parsed Model IDs
A regex that derived generation and tier from Gemini model IDs broke quietly once Flash reached 3.6 while the top Pro stayed at 3.5. Here are the runnable probes, the attribution gap between regex-derived and registry-joined rollups, and the redesign that treats model IDs as opaque keys.
Don't Let the AI Studio Developer Log Be Your Source of Truth: A Two-Layer Way to Observe the Interactions API
The Interactions API developer log in AI Studio is a great triage lens, but it is not a system of record. Here is a two-layer observability design with self-hosted structured logging, reconciliation code, measured sink sizing across 100,000 records, and per-feature cost attribution from an indie operator’s point of view.
When a Batch Job Sat in RUNNING for Half a Day: Field Notes on Catching Stalls Early with Per-State Dwell Budgets and Record Reconciliation
When a Gemini Batch job stalls quietly under the shadow of the 24-hour SLA, per-state dwell-time budgets and submitted-vs-completed record reconciliation let you name the stall early. Field notes with real operational numbers.
When Gemini API Leaks Japanese Into Your English Output Once in a While — Field Notes on Measuring the Contamination Rate and Tightening It in Stages
You told Gemini to answer in English, and 3 out of 100 runs slip a Japanese sentence into the tail. Here is why you cannot stop that 'once in a while', and a production pattern that measures the contamination rate as an SLO and tightens it with graded recovery, with working code.
When the Gemini Review Bot in Your CI Quietly Stops Earning Its Keep — Rebuilding Trust with Coverage and Actioned-Rate Metrics
A Gemini-powered PR review bot in GitHub Actions degrades without ever throwing an error. Field notes on catching diff truncation, model alias drift, and swallowed parse failures with one-line JSON logs and an actioned-rate metric.
Folding Scattered Call Sites Into One Front Door: Migrating to the Interactions API for Automation
With the Interactions API now generally available, Gemini's calls can settle behind a single entry point. Here is a migration design for folding scattered call sites — generateContent, Batch, and homegrown agent loops — into one front door without breaking anything, complete with a working adapter layer.
When Your Gemini Agent Has Three Tool Routes and Quietly Picks the Wrong One
Put Function Calling, Code Execution, and Grounding into one agent and the model will sometimes choose the wrong route, while the output still looks perfectly plausible. Here is how I instrument route selection and correct it with phase separation and verification gates, with working code.
The Morning a Managed Agent Stalled and Left No Trace — Building a Run-Observability Layer Outside the Sandbox
With Gemini Managed Agents, the sandbox lives on Google's side, so when a run stalls there is nothing left in your own logging stack. This is a working TypeScript design for an outside observability layer that taps stream events into a ledger, detects silent stalls, and folds runs into readable postmortems.
When Gemini's Safety Filter Silently Drops Legitimate Output — Field Notes on Catching False Positives Without Turning Everything Off
Field notes on handling Gemini API false positives in production without disabling every category. Separating input blocks from output blocks, instrumenting per-category false-positive rates, and recovering by relaxing only the offending category.
When Gemini API Quietly Dies on the Edge from Subrequest Limits — Field Notes on Budgeting What's Left
Running Gemini API on Cloudflare Workers is calm until traffic rises or a tool chain deepens, and then it fails on the subrequest limit. Here are the instrumentation patterns I use to measure per-request consumption and treat it as a budget, drawn from the sites I run as an indie developer.