All Articles
Green Tests, Dead Production — How Recorded Fixtures Hide a Model Retirement, and a Freshness Gate to Catch It
A test suite that replays recorded API responses will sail straight past a model retirement. I reproduce the failure in a minimal setup and build a cassette freshness gate, with measured overhead.
Retrying a Retired Model Never Showed Up in Success Latency
With the image generation models shutting down soon, I rebuilt a mock server to see what my retry layer actually does when it hits a retired model ID. The damage landed in wall time and queue wait, and never touched the metric I was watching.
Measuring Update Policies for Memory Profiles: The Guard That Cost Me 16 Points of Accuracy
Memory profiles went GA in Memory Bank, making structured memory available to downstream code. I built three update policies and compared them under identical conditions. The one that looked obviously correct turned out to be the worst. Full harness code and the path to per-field TTLs.
Measuring a Guard in environment hooks: 46 Microseconds to Decide, 23 Milliseconds to Start
A record of building a destructive-command guard for Managed Agents environment hooks. A regex denylist let 9 of 20 dangerous commands through; argv parsing reached 100 percent detection. The startup cost turned out to be 500 times the decision cost.
When Sampling Parameters Were Deprecated, Diversity Broke Before Determinism Did
Deprecating temperature, top_p and top_k hurt the diversity-generating side of my pipeline, not the deterministic side. Counting real call sites, moving diversity to the input layer, and measuring effective diversity.
After the Vertex AI Rename, I Measured My Search Stack. Ranking Held. Aggregation Did Not.
When Vertex AI became Gemini Enterprise Agent Platform, I benchmarked my own docs search. Ranking barely moved. The exact-match layer quietly lost most of its results without raising a single error.
What Decided Our Cascade's Economics Wasn't the Escalation Rate
Putting Flash-Lite in front of 3.6 Flash and escalating only the hard items looks like an easy win. Estimating it from the escalation rate alone will mislead you. Here is the break-even solved with the escalated subset's output length in the equation.
A 17% Drop in Output Tokens Sounded Big. Then I Decomposed the Bill
Output pricing fell 16.67% and output tokens fell about 17%. Adding those numbers does not give you the savings. Here is an exact price/volume/cross-term decomposition, plus a loop measurement where the savings rate went down instead of up.
When Version Numbers Stopped Meaning Generations: Rebuilding Cost Attribution That Parsed Model IDs
A regex that derived generation and tier from Gemini model IDs broke quietly once Flash reached 3.6 while the top Pro stayed at 3.5. Here are the runnable probes, the attribution gap between regex-derived and registry-joined rollups, and the redesign that treats model IDs as opaque keys.
Retiring Prompt Assets That Refuse to Die — A Ledger for Paths Static Analysis Cannot See
A prompt I had supposedly consolidated away was still running daily. Here is the retirement ledger I built to cross-check static references, flag defaults, observation windows, model end-of-life dates, and cache effectiveness.
Wiring a Security-Focused Model Into a Solo Developer's Audit — The Extraction Layer and Fingerprints That Stop Re-Reporting
A three-layer design that extracts outbound-request sinks with the AST, then accepts a model's hypothesis only when the reproduction actually runs. Four fingerprint schemes measured, including the collision that hides a real finding behind a safe twin.
The Day I Stopped Tracking gemini-flash-latest: Batch Design That Survives Silent Model Swaps
A silent model swap pushed my batch rejection rate from 2.1% to 9.8% overnight. The pinning-plus-canary design I moved to, with the harness and numbers.