Migrating from Gemini 2.5 Pro to 3.2 Pro in 7 Days — A Production Playbook for Compatibility Testing, Output Diff Scoring, and Rollback Design
A 7-day playbook for moving production systems from Gemini 2.5 Pro to 3.2 Pro: compatibility testing, LLM-as-Judge scoring, shadow traffic, and rollback.
Don't Let Your Gemini Prompts Silently Rot — A Practical Regression Testing Playbook with Pytest
Ever tweaked a prompt and watched production quality quietly degrade? This article walks through testing Gemini API prompts with Pytest, combining snapshot tests and LLM-as-Judge to catch regressions automatically — all from the perspective of an individual developer running things solo.
I Asked Gemini to Grade My App Store Screenshots. Everything Scored 78–85.
Ask Gemini Vision to grade App Store screenshots out of 100 and good candidates and deliberately broken ones both land at 78–85. Here is how I measured the judge's discrimination power, dropped absolute scoring, and rebuilt it as debiased pairwise comparison.
When the Default Model Silently Upgrades: Catching Prompt Regressions in Numbers
Gemini 3.5 Flash is now the default and you can no longer turn it off. Assuming your responses can shift without you touching the prompt, here is how to bundle prompt, model, and sampling into one variant and catch regressions with canaries and an LLM judge — in working code.
Switching Image Models Quietly Degrades Quality — A Gate That Catches It Without Manual Review
When you move image generation from preview to GA models, the API keeps returning 200 and quality slips silently. This is the three-layer gate I built to detect that drift without staring at every image: deterministic property checks, multimodal embedding similarity, and a Gemini judge, wired together in Python with thresholds and a cutover procedure.
Designing a Continuous Quality Monitoring Pipeline for the Gemini API
A practical, indie-developer-friendly design for a Gemini API evaluation pipeline that catches silent quality regressions using a Golden Dataset and a multi-aspect LLM-as-Judge, with full code and real cost numbers.
Building a RAG Evaluation Framework with Gemini API: RAGAS, LLM-as-Judge, and Custom Metrics Production Masterclass
Complete guide to building a quantitative RAG evaluation framework using RAGAS, LLM-as-Judge with Gemini API, and custom domain metrics — including CI/CD integration and production monitoring.
Building an LLM-as-Judge Evaluation Pipeline with Gemini — Production-Grade Design and Implementation
A practical guide to building an LLM-as-Judge evaluation pipeline using Gemini 2.5 Pro / 3 Pro as the judge. Covers Pointwise / Pairwise judging, bias mitigation, human-correlation measurement, and cost optimization, with working Python code for production use.
Building a Prompt Evaluation & Optimization Pipeline with Gemini API — Automated Quality Scoring with LLM-as-Judge
Learn how to build a prompt evaluation pipeline using Gemini API. Covers the LLM-as-Judge pattern, A/B testing prompts, automated quality scoring, and cost-quality optimization for production systems.