Defending Gemini API Responses with Schema Validation — Never Let Unexpected Formats Reach Production

When I started using Gemini API's structured output mode (responseSchema), I genuinely thought I was done worrying about JSON.parse exceptions. During development, responses come back in the right shape almost 100% of the time. Then a few weeks after going live, errors began clustering: the JSON was structurally valid, but enum values had drifted to strings I had never seen.

Looking into it, Gemini's structured output guarantees the result is valid JSON, but it does not strictly enforce the constraints declared in the schema — enums, min/max bounds, required fields, patterns. Those are best-effort on the model side. At production volume, something always slips through.

This article documents the layered defense I now run on Gemini API responses. Validation with Zod or Pydantic, retry with repair prompts, and graceful degradation — at code level.

Three failure patterns that leak through structured output

The "unexpected format reaches production" incidents I have seen group cleanly into three patterns.

Drifted enum values. A field declared as ["positive", "negative", "neutral"] returns "mixed" or "slightly positive". Even with explicit prompt instructions like "do not return values outside the enum," the leak rate sits around 3%.

Numeric range escapes. A schema saying { "score": { "type": "number", "minimum": 0, "maximum": 100 } } occasionally returns 105 or -3. In my sentiment-analysis traffic this hits about 0.5%. Rare, but at production volume that is a handful of incidents per month.

Array length deviation. "Return the top 3 keywords" sometimes returns 2, sometimes 5. If the frontend assumes exactly 3, the layout breaks.

These are not Gemini bugs so much as essential limitations of today's structured output APIs. To run in production, you need to switch from "trusting the API contract" to "verifying that the contract was honored on every call."

The three-stage defense at a glance

The layered defense I run looks like this:

1. Stage 1: Schema validation (Zod/Pydantic) — does the response strictly conform?
2. Stage 2: Repair-prompt retry — if stage 1 failed, ask Gemini to regenerate with the error included
3. Stage 3: Graceful degradation — absorb leftover failures so the UI does not break

Each stage in turn.

Stage 1: Schema validation — pin the type with Zod or Pydantic

Even though Gemini receives a responseSchema, you also define the same schema on your side. It looks like double maintenance; for production defense it is essential.

TypeScript example with Zod:

import { z } from "zod";
 
const SentimentEnum = z.enum(["positive", "negative", "neutral"]);
 
const AnalysisResultSchema = z.object({
  sentiment: SentimentEnum,
  score: z.number().min(0).max(100),
  keywords: z.array(z.string()).length(3),
  summary: z.string().min(10).max(500),
});
 
type AnalysisResult = z.infer<typeof AnalysisResultSchema>;
 
function parseGeminiResponse(rawJson: string): AnalysisResult {
  const parsed = JSON.parse(rawJson);
  return AnalysisResultSchema.parse(parsed);
}

The critical point is to keep the schema sent to Gemini and the Zod schema in lockstep. I generate one from the other so they cannot drift. Zod can emit JSON Schema via zod-to-json-schema, and that goes straight to Gemini:

import { zodToJsonSchema } from "zod-to-json-schema";
 
const responseSchemaForGemini = zodToJsonSchema(AnalysisResultSchema, {
  target: "openApi3",
});
 
const result = await model.generateContent({
  contents: [{ parts: [{ text: prompt }] }],
  generationConfig: {
    responseMimeType: "application/json",
    responseSchema: responseSchemaForGemini as any,
  },
});

With Pydantic the same pattern applies — model_json_schema() produces the JSON Schema you hand to Gemini.

Stage 2: Repair-prompt retry

When stage 1 throws a ZodError, do not jump straight to fallback. Hand the error back to Gemini and let it fix its own output. I call this the "repair prompt."

async function generateWithRepair(
  prompt: string,
  maxAttempts: number = 2,
): Promise<AnalysisResult> {
  let lastError: z.ZodError | null = null;
  let lastRawResponse: string | null = null;
 
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    const finalPrompt = lastError
      ? buildRepairPrompt(prompt, lastRawResponse!, lastError)
      : prompt;
 
    const raw = await callGemini(finalPrompt);
 
    try {
      return parseGeminiResponse(raw);
    } catch (err) {
      if (err instanceof z.ZodError) {
        lastError = err;
        lastRawResponse = raw;
        continue;
      }
      throw err;
    }
  }
 
  throw new SchemaValidationFailed(lastError!, lastRawResponse!);
}
 
function buildRepairPrompt(
  originalPrompt: string,
  lastResponse: string,
  error: z.ZodError,
): string {
  const issues = error.issues
    .map((i) => `- ${i.path.join(".")}: ${i.message}`)
    .join("\n");
 
  return `${originalPrompt}
 
Your previous response was:
 
\`\`\`json
${lastResponse}
\`\`\`
 
It violated the schema. Please fix the following issues and produce a
new response that strictly satisfies the same schema:
 
${issues}
`;
}

In my measurements this is highly effective: the 3% stage-1 failure rate drops to 0.3% after one repair attempt, and to 0.05% after two. Cost rises only on failures, so the monthly cost increase is in the low single-digit percent range.

A caveat: including the previous response in the prompt expands context. I cap retries at two — a third miss falls through to stage 3.

Stage 3: Graceful degradation — the last line for the UI

A handful of responses per month will fail even after stage 2. Throwing an exception there means the user sees a generic error screen, which is poor UX.

The fallback handler absorbs schema violations into a usable shape so the UI stays intact. I define per-field strategies:

function fallbackForSchemaFailure(
  raw: unknown,
  error: z.ZodError,
): AnalysisResult {
  const partial = raw as Partial<AnalysisResult>;
 
  return {
    sentiment: SentimentEnum.safeParse(partial.sentiment).success
      ? (partial.sentiment as "positive" | "negative" | "neutral")
      : "neutral",
    score: clamp(typeof partial.score === "number" ? partial.score : 50, 0, 100),
    keywords: Array.isArray(partial.keywords)
      ? partial.keywords.slice(0, 3).concat(Array(3).fill("")).slice(0, 3)
      : ["", "", ""],
    summary: typeof partial.summary === "string"
      ? partial.summary.slice(0, 500).padEnd(10, " ")
      : "(summary unavailable)",
  };
}

The principle here is not "throw broken values away and fill with the median," but "preserve as much as possible." Sentiment of "slightly positive" is rejected by stage 1, but rounded to "positive" here. A score of 105 clamps to 100.

Crucially, every fallback gets logged. Otherwise you risk the day where every response is silently in fallback mode:

logger.warn("schema_fallback_used", {
  task_id: taskId,
  zod_issues: error.issues.map((i) => ({
    path: i.path.join("."),
    code: i.code,
  })),
});

I pipe these to Honeycomb / Datadog and alert when the fallback rate exceeds 1%.

Schema management — preventing the double-definition trap

Running this defense forces three places to stay in sync per task: Zod/Pydantic, the schema sent to Gemini, and the fallback handler. Manual sync drifts.

Three rules I follow:

Treat the Zod/Pydantic schema as the single source of truth. The JSON schema for Gemini is always generated; the fallback handler reads field names from a helper that walks the same schema.

Centralize schemas under src/schemas/gemini/ with one file per task and filenames matching task names — sentiment_analysis.ts, keyword_extraction.ts, etc.

Keep schema changes backward-compatible. New fields start optional, enum values can be added but never removed, and so on. This rule alone has saved me from production incidents during incremental rollouts.

What one month looked like in the numbers

Running this defense for a month against my own traffic produced these numbers, for reference:

• Stage 1 schema validation failure rate: ~2.8%
• Stage 2 repair-prompt recovery rate: 89% (the remaining 11% reach stage 3)
• Stage 3 fallback firing rate: ~0.3%
• User-facing "error" screens shown in production: 0

A 0.3% stage-3 rate at a few thousand requests per month means dozens of responses per month that would have been exceptions are quietly absorbed. The difference for the user is the difference between a working app and an error toast.

The initial build is half a day to a day. After that, schema changes and applying the pattern to new tasks become routine. If you have been frustrated by the "Gemini structured output that occasionally breaks" problem, start by writing one Zod or Pydantic schema and implementing only stage 1 — the rest of the defense will fall into place from there.