When Gemini's Structured Output Quietly Drifts From Your Schema — Field Notes on Measuring Validation and Retries

Split the cause with finish_reason

Before you touch resp.text at all, confirm the response actually finished. Most structured-output breakage happens before Pydantic ever sees the text.

class StructuredError(Exception):
    def __init__(self, reason: str):
        self.reason = reason
        super().__init__(reason)
 
def extract_text(resp) -> str:
    """Separate finish_reason and empty text, returning a labeled cause."""
    if not resp.candidates:
        raise StructuredError("no_candidates")
 
    cand = resp.candidates[0]
    fr = cand.finish_reason.name if cand.finish_reason else "UNKNOWN"
 
    if fr == "MAX_TOKENS":
        # Output was too long and got cut → JSON is guaranteed broken
        raise StructuredError("max_tokens")
    if fr == "SAFETY":
        raise StructuredError("safety_block")
    if fr != "STOP":
        raise StructuredError(f"finish_{fr.lower()}")
 
    if not resp.text:
        raise StructuredError("empty_text")
 
    return resp.text

Retrying a MAX_TOKENS cutoff as if it were schema drift is wasted effort. There you should raise max_output_tokens or split the output; re-firing the same prompt won't change a thing. SAFETY is the same — it needs an input rethink, not a retry. Putting finish_reason at the front of your branching cleanly separates "failures worth retrying" from "failures retrying can't fix," and that alone cuts a lot of wasted retry budget.

Don't just retry — feed the error back

Schema-shaped but wrong values, or slightly malformed JSON — this kind of failure clears far more easily if you tell the model what went wrong than if you resend the same prompt. Pydantic's ValidationError carries an explanation that's readable to both humans and models, so hand it straight back in the next prompt.

import json, time
from typing import Type, TypeVar
from pydantic import BaseModel, ValidationError
 
T = TypeVar("T", bound=BaseModel)
 
def generate_structured(prompt: str, model_class: Type[T],
                        model: str = "gemini-2.5-flash",
                        max_retries: int = 3) -> T:
    current = prompt
    for attempt in range(max_retries):
        try:
            resp = client.models.generate_content(
                model=model,
                contents=current,
                config=types.GenerateContentConfig(
                    response_mime_type="application/json",
                    response_schema=model_class,
                ),
            )
            text = extract_text(resp)          # finish_reason branch
            obj = model_class.model_validate_json(text)
            METRICS.record_success()
            return obj
 
        except StructuredError as e:
            METRICS.record_failure(e.reason)
            if e.reason in ("max_tokens", "safety_block"):
                raise                          # abort failures retrying can't fix
        except (ValidationError, json.JSONDecodeError) as e:
            METRICS.record_failure("validation")
            # Feed the error back and ask for a fix
            current = (
                f"{prompt}\n\n"
                f"Your previous output was invalid for this reason. Fix it and "
                f"return only JSON that strictly follows the schema:\n{e}"
            )
        time.sleep(1.5 * (attempt + 1))        # exponential backoff (also rate-limit friendly)
 
    raise StructuredError("exhausted_retries")

The corrected retry is noticeably effective. In my runs, a plain retry would often repeat the same failure on the second attempt, whereas feeding back the ValidationError body got most cases through on the second try. The model may overlook "rating is an integer from 1 to 5" in the first prompt, but pointed at the specific failure, it tends to comply. Meanwhile max_tokens and safety_block raise early and leave the loop, so retry budget isn't burned on failures that won't heal.

Landmines you only learn by stepping on them

These are the pitfalls the docs don't shout about but you hit in production. I've tripped over each at least once.

Union types break in schema conversion. A type like Union[str, int] converts to JSON Schema's anyOf, which Gemini doesn't interpret reliably. Narrow to one type, express "may be absent" as Optional[str] = None, and split numbers into their own field. The more ambiguity you leave in the schema, the higher the drift rate.

Pydantic's strict=True backfires against an LLM. In strict mode, dropping "85" into an int is rejected on the spot. But LLMs frequently return numbers as strings, so the default coercing behavior fits reality better. This is a place to favor pass-rate over type purity.

Fold nesting down to two levels. Past three levels, Gemini starts "interpreting" the structure — flattening it on its own or renaming fields. Rather than grabbing a deep tree in one call, a flat model or two calls came out far more stable.

Close the escape hatches with Enum. Allow a free-form str and your intended "positive" comes back as "Positive" or a translated variant. Inheriting str, Enum to enumerate the allowed values removes the room for unexpected strings entirely.

As an indie developer classifying wallpaper-app reviews with Gemini, I lost half a day to exactly this strict=True trap. My local test data only ever contained integers, so it sailed through, and only in production did real user reviews pull in string-valued ratings, quietly stalling the aggregation batch overnight. I hadn't instrumented the failure rate, so I only noticed from the gap in the next morning's data. Since then I always wire up rate measurement and per-cause counts before anything else on structured output.

The one move to make first

If you're putting structured output into production, before any retry design, add just one thing: instrumentation that counts success and failure by cause. Once the failure rate is visible, whether to add a finish_reason branch, drop strict, or fold the nesting becomes a decision driven by numbers instead of guesswork. Layer the retry function from this article on once you know where to fix — it'll be in time.