How I Decide Between 3.8 Flash and 3.7 Flash — A Fixed Set of 20 Questions
After swapping a model ID to a newer generation, the same input can start producing answers that feel split. Here is the 20-question evaluation set I keep, how I run both generations on equal footing, and the two numbers I use to decide.
Judging Gemma 4 and Nemotron 3 Nano Omni on 100 of My Own Images, Not a Benchmark Score
Heron-Bench and JMMMU headline scores are the wrong input for an adoption decision on local Japanese multimodal models. Using a wallpaper classifier as the case, here is how to build a 100-image eval set, weight errors by what they actually cost, and catch regressions when you re-quantize.
Gemini API × Langfuse — A Production Playbook for LLM Observability
A practical, production-grade guide to wiring Gemini API into Langfuse — tracing architecture, cost attribution, LLM-as-Judge on live traffic, PII masking, and sampling — with runnable code.
A Japanese query won't surface its English twin — when embeddings notice language before meaning
Embed a translation pair with gemini-embedding-2 and the two halves won't be nearest neighbours, because language itself inflates similarity. Here is how I measured cross-lingual recall using translation pairs as ground truth, and what happened when I subtracted the language centroid.
Building a RAG Evaluation Framework with Gemini API: RAGAS, LLM-as-Judge, and Custom Metrics Production Masterclass
Complete guide to building a quantitative RAG evaluation framework using RAGAS, LLM-as-Judge with Gemini API, and custom domain metrics — including CI/CD integration and production monitoring.
Building an LLM-as-Judge Evaluation Pipeline with Gemini — Production-Grade Design and Implementation
A practical guide to building an LLM-as-Judge evaluation pipeline using Gemini 2.5 Pro / 3 Pro as the judge. Covers Pointwise / Pairwise judging, bias mitigation, human-correlation measurement, and cost optimization, with working Python code for production use.
Quietly Catching Wrong Answers in Your Gemini-Powered App — A Production Auto-Eval Loop
Running Gemini in production eventually shows you responses that are 'kind of wrong.' I want to catch them before users do. This is the exact auto-eval loop I run over live traffic, with the prompts I use and the mistakes I had to learn my way through.
Building a Prompt Evaluation & Optimization Pipeline with Gemini API — Automated Quality Scoring with LLM-as-Judge
Learn how to build a prompt evaluation pipeline using Gemini API. Covers the LLM-as-Judge pattern, A/B testing prompts, automated quality scoring, and cost-quality optimization for production systems.