◉GEMINI LABJP
●CHANGELOG — No new Gemini API changelog entries since 11/19; the stable CLI is still v0.70.0●2.5 ROUTE — From 12/1, gemini-2.5-pro reroutes to 3.8 Flash and 2.5-flash to 3.5 Flash-Lite. 8 days left●SPEND CAP — Developers are asking for a hard spend limit on the API. Set budget alerts and decide how you would pause a key●CARBON — Gemini 4 Carbon launch pricing ends 11/30, 7 days left●OMNI 1.2 — On 12/3, gemini-omni-1.1-flash switches to 1.2 automatically●/SHELLS — /shells still appears in the docs but was removed. If a command fails, check whether it still exists●CHANGELOG — No new Gemini API changelog entries since 11/19; the stable CLI is still v0.70.0●2.5 ROUTE — From 12/1, gemini-2.5-pro reroutes to 3.8 Flash and 2.5-flash to 3.5 Flash-Lite. 8 days left●SPEND CAP — Developers are asking for a hard spend limit on the API. Set budget alerts and decide how you would pause a key●CARBON — Gemini 4 Carbon launch pricing ends 11/30, 7 days left●OMNI 1.2 — On 12/3, gemini-omni-1.1-flash switches to 1.2 automatically●/SHELLS — /shells still appears in the docs but was removed. If a command fails, check whether it still exists
Articles/API / SDK
◈ API / SDK/2026-05-29Advanced

Layering Gemini API Response Caches in Three Tiers — How I Split Memory, Redis, and Context Cache

Notes from running a three-tier cache (in-memory, Redis, Gemini Context Cache) in front of the Gemini API for six weeks across a wallpaper app — actual hit rates, billing impact, and the invalidation traps that ate me alive.

gemini-api291cacheredis3context-cachearchitecture16cost-optimization30

✦ Premium Article

I'm Hirokawa, an indie developer based in Tokyo. Since 2014 I've been shipping wallpaper and calming-style apps under the name Dolice — they've crossed 50M cumulative downloads, and as I started moving generation work to the Gemini API the bill quietly started to bite. AdMob revenue was getting routed into Gemini, then back into the next ad impression, and the loop kept getting thinner.

In April 2026 I rebuilt the caching layer around three tiers: in-process memory, Redis, and Gemini's own Context Cache. Each one owns a different slice of the problem. Six weeks of production traffic later, here's the part that surprised me, the parts that hurt, and the layout I've settled on.

Why three tiers and not just one

I started with Redis only, assuming a single layer would be enough. Then I actually looked at the traces and realized three distinct redundancies were happening at three different time scales.

  • L1 — In-process LRU (4 MB, 30s TTL). Absorbs sub-second duplicates: the same user re-rendering the same screen, page transitions firing the same prompt twice, that kind of thing. Lives in the Cloudflare Workers isolate, so cold starts wipe it
  • L2 — Redis (Upstash, 24h TTL). Catches "byte-identical responses" shared across users — promptly hashed with the model name, temperature, top_p, thinking_budget, and response_mime_type
  • L3 — Gemini Context Cache (1h–24h TTL). Stores the long system prompt and the reference material itself on Gemini's side, so we don't pay to re-send 50,000 tokens of brand guidelines every request

L1 is the fastest and cheapest, L3 is slower per request but directly cuts the input-token bill. Splitting them is really about keeping speed, cross-user sharing, and token reduction as independent dimensions.

Six weeks of measured numbers

Numbers below are pulled from Cloudflare Analytics and the Gemini billing dashboard between April 12 and May 24, 2026. About 12,140 requests per day on average, with a peak of 580 req/min.

  • L1 hit rate: 67.2% — average response time 4 ms
  • L2 hit rate: 22.4% — average 11 ms
  • L3 hit rate: 4.1% — Gemini still serves the request, but input tokens are billed at roughly 25% of the uncached rate
  • All-tier miss: 6.3% — fresh generation, full latency and full bill

Net effect: actual Gemini calls dropped from 12,140/day to 765/day. The month's bill went from $124.30 to $38.71, a 68.8% reduction. At the AdMob eCPM I was seeing that period ($1.42), that's about ¥270/day of breathing room. A single ad's worth of margin recovered — small, but exactly the kind of inch that matters when you're a solo developer trying to nudge the break-even line down.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN
✦Six weeks of production data showing 67% / 22% / 4% hit rates across the three tiers, plus a Gemini bill that dropped from $124/month to $38/month on ~12,000 daily requests
✦Why Gemini Context Cache (with its 32,768-token minimum) belongs at a different layer than Redis, and how to size each tier so they reinforce instead of duplicate each other
✦A write-through plus stale-while-revalidate pattern that finally stopped the 'wrong vocabulary set leaking into yesterday's category' bug we ran into when we tried to flush Redis without invalidating Context Cache
Secure payment via Stripe · Cancel anytime
✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

or
Unlock all articles with Membership →
Share

Thank You for Reading

Gemini Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

  • ✦Copy-paste ready implementation code
  • ✦New advanced guides published daily
  • ✦$5/mo or $10 for lifetime access
View Membership →

Related Articles

◈ API / SDK2026-07-23
When to Hand Conversation State to the Server: previous_interaction_id vs. Pruning History Yourself
With the Interactions API, passing previous_interaction_id lets the server hold conversation state so you stop resending history. But a large tool output that lands mid-conversation can't be pruned afterward, and every later turn drags its weight. Here is a branching design that mixes server-side state with client-side pruning, plus a thin, working Python wrapper.
◈ API / SDK2026-04-07
Gemini API Semantic Router: Implementation Notes for Splitting Flash and Pro Smartly
Implementation notes for building a production-grade semantic router that automatically dispatches Gemini queries between Flash and Pro. Includes Python and TypeScript working code, a two-stage design pattern, and seven implementation insights from running it inside an indie wallpaper app.
◈ API / SDK2026-05-14
Controlling thinking_budget in Gemini 2.5 Pro — Cut Costs by 70% Without Sacrificing Reasoning Quality
Leaving thinking_budget unset in Gemini 2.5 Pro leads to unexpected costs. This guide covers task-level budget design, dynamic control, and production monitoring with working Python code.