Building Serverless AI Apps with Firebase Genkit and Gemini — An Implementation Notebook

Your first Flow — pin inputs and outputs with zod

A Genkit Flow bundles "input schema, processing, output schema" into one function. Writing the schema in zod lets the Dev UI generate an input form for you, and keeps execution traces typed.

// src/flows/greeting.ts
import { z } from "genkit";
import { ai } from "../genkit";
 
export const greetingFlow = ai.defineFlow(
  {
    name: "greeting",
    inputSchema: z.object({
      name: z.string(),
      language: z.enum(["en", "ja"]),
    }),
    outputSchema: z.string(),
  },
  async ({ name, language }) => {
    const prompt =
      language === "en"
        ? `Write a short, friendly greeting for ${name}.`
        : `${name}さんへの、短くて親切な挨拶を日本語で書いてください。`;
 
    const { text } = await ai.generate({ prompt });
    return text;
  },
);

Verify locally from the Dev UI.

# Start the Dev UI (try flows at http://localhost:4000)
npx genkit start -- npx tsx --watch src/index.ts

The important detail here is reading text via destructuring from the return of ai.generate() — not a method call like the response.text() you may see in older samples. In Genkit 1.x the return value is a property, and that difference is the first thing that trips up a copy-and-paste.

Multi-step Flow — fold document analysis into one function

Folding several preprocessing steps and a generation call into a single Flow means the caller only has to pass input. Here we take a document and return a summary, sentiment, or keywords.

// src/flows/documentAnalysis.ts
import { z } from "genkit";
import { ai } from "../genkit";
 
export const documentAnalysisFlow = ai.defineFlow(
  {
    name: "documentAnalysis",
    inputSchema: z.object({
      documentText: z.string(),
      analysisType: z.enum(["summary", "sentiment", "keywords"]),
    }),
    outputSchema: z.object({
      analysisType: z.string(),
      result: z.string(),
    }),
  },
  async ({ documentText, analysisType }) => {
    // Step 1: clamp the input to a safe length
    const cleaned = documentText.trim().slice(0, 5000);
 
    // Step 2: prepare a prompt per analysis type
    const prompts: Record<string, string> = {
      summary: `Summarize this document in three sentences:\n${cleaned}`,
      sentiment: `Rate the sentiment as negative / neutral / positive:\n${cleaned}`,
      keywords: `List the five most important keywords:\n${cleaned}`,
    };
 
    // Step 3: generate with Gemini and return a structured result
    const { text } = await ai.generate({ prompt: prompts[analysisType] });
    return { analysisType, result: text };
  },
);

Clamping the input at 5,000 characters guards against an unexpectedly long body spiking your token cost. Set the limit to match your documents, but keeping it on the outside removes one class of accidents.

Tool Calling and agents — hand tool choice to the model

In Genkit you define a tool with ai.defineTool and simply pass it to generate; the model decides when to call it. You don't build a special class to get an agent.

// src/tools/weather.ts
import { z } from "genkit";
import { ai } from "../genkit";
 
export const getWeather = ai.defineTool(
  {
    name: "getWeather",
    description: "Return the current weather for a given city",
    inputSchema: z.object({ city: z.string() }),
    outputSchema: z.object({
      city: z.string(),
      temperature: z.number(),
      condition: z.string(),
    }),
  },
  async ({ city }) => {
    // In production, call a real weather API here
    return { city, temperature: 25, condition: "Sunny" };
  },
);

// src/flows/assistant.ts
import { z } from "genkit";
import { ai } from "../genkit";
import { getWeather } from "../tools/weather";
 
export const assistantFlow = ai.defineFlow(
  {
    name: "assistant",
    inputSchema: z.object({ question: z.string() }),
    outputSchema: z.string(),
  },
  async ({ question }) => {
    const { text } = await ai.generate({
      prompt: question,
      tools: [getWeather],
      system: "You are a capable assistant. Use tools only when needed, and answer concisely.",
    });
    return text;
  },
);

The more tools you add, the more room the model has to pick the wrong one at the wrong time. A tool's description is the model's only basis for that decision; vague wording drives up misfires. I've written up how to measure and fix this in instrumenting agent tool misselection.

RAG — connect to Firestore Vector Search

In RAG you turn the query into an embedding, retrieve the nearest documents, and answer grounded only in that context. Genkit handles embeddings through ai.embed.

// src/flows/rag.ts
import { z } from "genkit";
import { ai } from "../genkit";
import { googleAI } from "@genkit-ai/googleai";
 
export const ragFlow = ai.defineFlow(
  {
    name: "documentRAG",
    inputSchema: z.object({ query: z.string(), topK: z.number().default(3) }),
    outputSchema: z.string(),
  },
  async ({ query, topK }) => {
    // 1. Turn the query into an embedding vector
    const [embedding] = await ai.embed({
      embedder: googleAI.embedder("text-embedding-004"),
      content: query,
    });
 
    // 2. Retrieve nearest documents via Firestore Vector Search
    const docs = await searchSimilar(embedding.embedding, topK);
 
    // 3. Build a prompt grounded only in the retrieved context
    const context = docs.map((d) => `- ${d.content}`).join("\n");
    const { text } = await ai.generate({
      prompt: `Answer concisely, grounded only in the reference material below. If there is no basis, say so.\n\nReference:\n${context}\n\nQuestion: ${query}`,
    });
    return text;
  },
);
 
// The Firestore nearest-neighbor search lives in a separate article
async function searchSimilar(
  _embedding: number[],
  _topK: number,
): Promise<Array<{ content: string }>> {
  return [];
}

The actual nearest-neighbor search and re-indexing drift are covered in Firestore × Gemini embedding re-embedding drift notes, and the multimodal extension in building a Gemini multimodal RAG system. For the relationship between embedding dimensions and vector-DB cost, see cutting vector-DB cost with Matryoshka dimension reduction.

Deploy to Cloud Functions — wrap it thinly with onCallGenkit

A Flow deploys to Firebase Functions as-is. onCallGenkit keeps the wiring for auth, streaming, and App Check short.

// functions/src/index.ts
import { onCallGenkit } from "firebase-functions/https";
import { defineSecret } from "firebase-functions/params";
import { greetingFlow } from "./flows/greeting";
import { assistantFlow } from "./flows/assistant";
 
// Inject the API key from Secret Manager
const apiKey = defineSecret("GOOGLE_API_KEY");
 
export const greeting = onCallGenkit({ secrets: [apiKey] }, greetingFlow);
export const assistant = onCallGenkit({ secrets: [apiKey] }, assistantFlow);

The deployment steps:

# 1. Install the Firebase CLI
npm install -g firebase-tools
 
# 2. Register the API key in Secret Manager (keep it out of code)
firebase functions:secrets:set GOOGLE_API_KEY
 
# 3. Deploy only Functions
firebase deploy --only functions
 
# 4. Verify
firebase functions:list

Receiving the API key through defineSecret is the key point. Inlining an environment variable creates a leak path into your repo or logs. Keeping it in Secret Manager also lets you rotate keys without a code change. For larger production workloads, deploying to Vertex AI Agent Engine is another option.

Cost optimization — model routing and making spend visible

Serverless billing quietly balloons through a single flow if you let it. The first thing I add is routing that sends work to a model based on difficulty.

// src/flows/routed.ts
import { z } from "genkit";
import { ai } from "../genkit";
import { googleAI } from "@genkit-ai/googleai";
 
const FLASH = googleAI.model("gemini-2.5-flash");
const PRO = googleAI.model("gemini-2.5-pro");
 
export const routedFlow = ai.defineFlow(
  {
    name: "routed",
    inputSchema: z.object({ prompt: z.string(), hard: z.boolean() }),
    outputSchema: z.string(),
  },
  async ({ prompt, hard }) => {
    // Send only hard tasks to Pro; handle the rest on Flash
    const { text } = await ai.generate({ model: hard ? PRO : FLASH, prompt });
    return text;
  },
);

Alongside that, log call counts and duration per flow. Watching only the grand total hides which flow is driving the bill. On my own setup I was running a simple classification task on Pro for a while; dropping it to Flash roughly halved that flow's model cost. My broader take on staged cost control is in cost guardrails for indie developers.

Stop guessing at cost — record usage per flow and read the itemized bill

The trouble that showed up right after I added routing was that I couldn't tell whether it was working. Billing arrives per project, not per Flow. When the total drops, there's no way to separate "routing paid off" from "that was just a quiet week."

So I started writing out the usage that comes back from ai.generate(), one line per call, tagged with the flow name.

// src/usage.ts
import { appendFileSync } from "node:fs";
import { ai } from "./genkit";
 
type GenArgs = Parameters<typeof ai.generate>[0];
 
// Wrap generate with a flow name and leave usage behind as one JSON line
export async function generateWithUsage(flow: string, args: GenArgs) {
  const startedAt = Date.now();
  const res = await ai.generate(args);
  const u = res.usage ?? {};
  const record = {
    ts: new Date().toISOString(),
    flow,
    model:
      typeof args.model === "string" ? args.model : args.model?.name ?? "default",
    inputTokens: u.inputTokens ?? 0,
    outputTokens: u.outputTokens ?? 0,
    ms: Date.now() - startedAt,
  };
 
  if (process.env.USAGE_LOG) {
    appendFileSync(process.env.USAGE_LOG, JSON.stringify(record) + "\n");
  } else {
    // Files don't survive on Cloud Functions, so emit a structured log instead
    console.log(JSON.stringify({ severity: "INFO", genkitUsage: record }));
  }
  return res;
}

This is where I tripped once: I kept appending to a file the way I did locally. A Cloud Functions instance takes its filesystem with it when it goes away, so those lines are not a log you can trust. In production I switched to emitting JSON through console.log and picking it up as a log-based metric in Cloud Logging. Locally I set USAGE_LOG, collect the jsonl, and aggregate it with the script below.

// scripts/usage-report.mjs
import { readFileSync } from "node:fs";
 
// Prices change — replace these with the numbers on your own billing page (USD per 1M tokens)
const PRICE = {
  "gemini-2.5-flash": { in: 0.3, out: 2.5 },
  "gemini-2.5-pro": { in: 1.25, out: 10.0 },
};
 
const rows = readFileSync(process.argv[2], "utf8")
  .trim()
  .split("\n")
  .map((line) => JSON.parse(line));
 
const agg = new Map();
for (const r of rows) {
  const key = `${r.flow}\t${r.model}`;
  const a = agg.get(key) ?? { calls: 0, in: 0, out: 0, ms: [] };
  a.calls += 1;
  a.in += r.inputTokens;
  a.out += r.outputTokens;
  a.ms.push(r.ms);
  agg.set(key, a);
}
 
const pct = (xs, p) => [...xs].sort((x, y) => x - y)[Math.floor(xs.length * p)] ?? 0;
 
for (const [key, a] of agg) {
  const [flow, model] = key.split("\t");
  const price = PRICE[model] ?? PRICE["gemini-2.5-flash"];
  const usd = (a.in / 1e6) * price.in + (a.out / 1e6) * price.out;
  const per1k = (usd / a.calls) * 1000;
  console.log(
    [
      flow.padEnd(18),
      `calls=${a.calls}`,
      `in/call=${Math.round(a.in / a.calls)}`,
      `out/call=${Math.round(a.out / a.calls)}`,
      `$${per1k.toFixed(2)} per 1,000 calls`,
      `p90 ${pct(a.ms, 0.9)}ms`,
      `p90 input ${pct(rows.filter((r) => r.flow === flow).map((r) => r.inputTokens), 0.9)} tok`,
    ].join("  "),
  );
}

Running my four flows on Flash produced the numbers below, calculated with the prices in that script.

What the table finally showed me is that the tail of the input distribution matters more than the average. documentAnalysis averages 1,850 input tokens, but its p90 is roughly 4,900 — about 2.6x the mean. The top ten percent of calls were producing something like thirty percent of that flow's monthly cost.

The 5,000-character truncation should have prevented that, so why did the tail keep growing? Because the cap only applied to the document that was passed in. The boilerplate around the prompt, and the retrieved context injected on the RAG path, were never counted. I now apply the cap to the assembled prompt as a whole and drop from the end of the retrieved context when it overflows.

Keep the price table in code as a constant. Once it lives in a spreadsheet, it stops getting updated, and you end up making decisions on six-month-old rates.

Cold starts and operation — the unglamorous tuning that paid off

The first wall in serverless was cold starts. The less often a flow is called, the slower its first invocation, and to a user that reads as a heavy app. Here are my own measurements.

The fixes are plain. Set minInstances to 1 on high-traffic paths to keep them warm, and move heavy initialization off module load and outside the request path. There's nothing flashy here, but perceived speed really does come down to this kind of tuning. Since minimum instances mean continuous billing, the indie compromise is to warm only the paths that genuinely need it.

Whether to set a minimum instance comes down to three questions for me.

1. Is this path part of the user's first interaction? If not, four to six seconds on the first call does little real damage
2. How far apart are the calls? If the logs show gaps of fifteen minutes or more as the norm, warmth won't survive on its own — nothing improves until you pay for a minimum instance
3. Does the monthly cost of holding one instance warm balance against that flow's model spend?

Question three flipped a decision for me. The lightweight greeting path costs about $0.26 per 1,000 calls. At a few thousand calls a month, the always-on charge for a minimum instance came out larger than the model spend itself. I dropped the minimum instance there and put a single warm instance on documentRAG instead. Warm every path out of a general desire for speed and the model cost you trimmed simply comes back under a different name.

Four things that passed in the Dev UI and broke in production

The local Dev UI is generous. It reads .env, fills schema defaults into the form, and never makes you think about regions. All of that generosity came due at once after deployment. Here are the four I actually hit.

Three steps have been enough to isolate every one of these.

1. Pull the logs for that specific function and check whether it fails only on the first call or on every call. That split separates wiring problems from cold-start problems
2. Failing every time means wiring. Work through secrets, region, and schema in that order — those three explained every case I ran into
3. Failing only the first time means initialization. Count what runs at module top level and see whether moving it behind the request changes anything

The region mismatch cost me half a day. The error wears a CORS face, so you keep suspecting the frontend. Checking the region in the functions list is a far shorter road.

Wrapping up — start small, and grow with the spend breakdown in hand

Genkit's value is that the framework is thin, letting you focus on the Gemini call. Start by running a single greeting Flow locally and deploying it. Once that one goes through, Tool Calling and RAG are the same way of writing, extended.

As a next step, record from day one the material you need to judge whether each flow warrants minInstances — its per-flow latency and call frequency. It's far easier to grow with that in hand than to add it later.

I hope this gives you a foothold for your own build. Thank you for reading.

To go deeper, see the Firebase Genkit documentation and the Gemini API documentation.