A common need when building agents is straightforward: call external APIs through function calling, then return the final answer in a fixed JSON shape so the frontend can render it deterministically. The first time I tried this in Gemini API, I passed tools and responseSchema to the same generateContent call and got back a terse 400 INVALID_ARGUMENT saying "Function calling is not enabled for models with response schema." The cause is reasonable once you understand it, but the message alone makes it look like you have to give up one feature.
This article walks through why Gemini API forbids that combination, and then presents three designs I have actually run in production where function calling and structured output coexist cleanly. By the end, you will be able to pick the right pattern for your own use case and implement it without trial and error.
Why You Cannot Set Both at Once
In Gemini API, both tools (function calling) and responseSchema (structured output) constrain the model's output. Internally the API allows only one of those constraints to be active per call, which is why specifying both yields:
google.api_core.exceptions.InvalidArgument: 400 Function calling is not
enabled for models with response_schema. Please use tools or
response_schema, but not both.
When tools is enabled, the model can return either plain text or a functionCall part. When responseSchema is set, the model is forced to emit text that conforms to the JSON schema you provided. The two modes prescribe different output shapes, so they cannot be active simultaneously.
Worth noting: the official Tools page quietly added a sentence in early 2026 stating "do not use tools together with responseSchema." If you are porting a sample written before that, it is easy to miss. When you hit this 400, double-check the latest doc rather than your old code first.
Pattern 1: Define a Single "Final Answer" Function
This is the lightest pattern and the one I reach for most often. Instead of using responseSchema, declare your final answer shape as just another function.
# Python 3.11 / google-genai >= 0.5.0
from google import genai
from google.genai import types
client = genai.Client(api_key="YOUR_API_KEY")
# 1) An external-API tool
get_weather = types.FunctionDeclaration(
name="get_weather",
description="Return current weather for the given city",
parameters={
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
)
# 2) Final answer schema, expressed as a function
respond_to_user = types.FunctionDeclaration(
name="respond_to_user",
description="Final reply to the user. Always call this last.",
parameters={
"type": "object",
"properties": {
"summary": {"type": "string"},
"temperature_c": {"type": "number"},
"next_action": {
"type": "string",
"enum": ["wear_jacket", "bring_umbrella", "none"],
},
},
"required": ["summary", "temperature_c", "next_action"],
},
)
config = types.GenerateContentConfig(
tools=[types.Tool(function_declarations=[get_weather, respond_to_user])],
tool_config=types.ToolConfig(
function_calling_config=types.FunctionCallingConfig(mode="AUTO")
),
)
response = client.models.generate_content(
model="gemini-2.5-flash",
contents="Suggest what to wear in Tokyo today",
config=config,
)
# response.candidates[0].content.parts[-1].function_call.args
# arrives in the respond_to_user shapeExpected behavior: the model first calls get_weather, you return the result, and on the next turn it calls respond_to_user with the structured arguments. Reinforce "always call respond_to_user as your final action" in the system prompt to prevent the model from replying in plain text.
A useful trick: switch tool_config.mode to "ANY" with allowed_function_names=["respond_to_user"] only on the final turn. That forces the model to emit just the structured answer, which removes a whole class of last-mile bugs.
Pattern 2: Split Tool Use and Formatting Into Two Calls
When the agent uses several tools and you want to be strict about the final JSON, separating the tool-execution phase from the formatting phase is cleaner and easier to reason about.
# 1) First call: tools only — finish the function-calling loop
tool_config = types.GenerateContentConfig(
tools=[types.Tool(function_declarations=[get_weather])],
)
turn1 = client.models.generate_content(
model="gemini-2.5-flash",
contents=user_input,
config=tool_config,
)
# Execute the functionCall, append the result, and loop until the agent stops
# 2) Second call: drop tools, set responseSchema, ask for the final shape
schema_config = types.GenerateContentConfig(
response_mime_type="application/json",
response_schema={
"type": "object",
"properties": {
"summary": {"type": "string"},
"temperature_c": {"type": "number"},
"next_action": {
"type": "string",
"enum": ["wear_jacket", "bring_umbrella", "none"],
},
},
"required": ["summary", "temperature_c", "next_action"],
},
)
final = client.models.generate_content(
model="gemini-2.5-flash",
contents=conversation_so_far + [
types.Content(
role="user",
parts=[types.Part(text="Return the final answer as JSON")],
)
],
config=schema_config,
)
import json
result = json.loads(final.text)You pay an extra round-trip in latency and tokens, but each phase has a clear job. Logging is easier, you can run the formatting phase on a cheaper model (Flash, for instance), and retries can be scoped to one phase. For most production assistants I build, this is my default choice.
Pattern 3: Free-Form Text Plus Pydantic Validation
If you want to keep the agent loop pure function calling but still need a strict schema on the final answer, a pragmatic compromise is to let the model return free-form text and validate the embedded JSON in your own code with Pydantic.
from pydantic import BaseModel, ValidationError, Field
from typing import Literal
import json, re
class WeatherAnswer(BaseModel):
summary: str
temperature_c: float = Field(ge=-50, le=60)
next_action: Literal["wear_jacket", "bring_umbrella", "none"]
raw = response.text # Final text after the function-calling loop
match = re.search(r"\{.*\}", raw, re.DOTALL)
try:
parsed = WeatherAnswer.model_validate_json(match.group(0))
except (ValidationError, AttributeError):
parsed = retry_with_schema(raw) # Fall back to Pattern 2You will get a typed WeatherAnswer object, with out-of-range temperatures or invalid enums failing fast as ValidationError. Even when the prompt says "return JSON only," models occasionally wrap it in prose, so always pair the regex extraction with a fallback that re-runs the formatting under responseSchema. If you want to dive into type-safe pipelines, the Pydantic-based type-safe Gemini API guide explains the validation strategy in more depth.
Choosing the Right Pattern
These patterns are not competing — they fit different shapes of work. Here is how I pick:
- One or two tools, fixed final shape: Pattern 1 is the shortest path. Schema and prompt live in the same file, which keeps reviews simple.
- Multi-step agent that needs a strict final JSON: Pattern 2. Splitting phases lets you swap models, scope retries, and add monitoring without contortions.
- Existing text pipeline you cannot rewrite: Pattern 3. With Pydantic in front, you can reach production quality without touching
responseSchema.
If you are debugging the function-calling loop itself, Gemini API Function Calling not working — troubleshooting and Stopping infinite tool-calling loops are good companions. For 400s caused by the schema itself, see Function Calling schema invalid_argument — causes and fixes.
Implementation Notes That Save Time in Production
A few details that always seem obvious in hindsight but cost time the first time around.
Validate the function declaration early. Gemini accepts an OpenAPI 3.0 subset, not the whole spec. additionalProperties, oneOf, and $ref are silently dropped or cause 400s depending on the SDK version. Before you wire the function into the agent loop, run a single generateContent call with one minimal contents value to confirm the model accepts the schema. Catching schema-validation failures here is much cheaper than chasing them through agent retries.
Keep the conversation history honest. Whatever pattern you pick, the parts you append to contents between turns must match what the model returned. With Pattern 1, that means appending the assistant's functionCall part as-is, then a functionResponse part that you generated. Reconstructing them from a database with only the text content stripped out causes subtle errors — the model loses track of which call you are responding to and may emit a duplicate call or fall back to plain text.
Set a turn budget. Even with explicit instructions, models sometimes call tools more often than you expect. A simple counter that breaks the loop after, say, eight turns and falls back to a templated response is the cheapest insurance against runaway costs. I add this in every production agent regardless of which pattern I am using.
Log the raw response. Especially during early development, dump the full response.candidates[0] JSON (or the equivalent in your SDK) to your logs. When debugging "why didn't the model call respond_to_user", being able to read the actual parts list — not just the parsed function call — makes the difference between a five-minute fix and an hour of guesswork.
Common Mistakes to Avoid
A short list of things I have done wrong, so you do not have to.
- Putting the response schema inside
description. Models do not enforce schemas mentioned in natural language. If you want a fixed shape, it must live inparameters(Pattern 1) orresponse_schema(Pattern 2). - Calling
respond_to_usermid-loop. If the model invokes the final-answer function before all required tool data is available, you end up with placeholder values. Make sure your system prompt makes it explicit that this function is the last action, never an intermediate one. - Mixing snake_case and camelCase in the schema. Within a single function declaration, stick to one naming style. Gemini will not raise an error, but the model has a much easier time generating consistent arguments when the schema is internally consistent.
- Ignoring
tool_config.mode. Many developers leave it at the default and wonder why the model occasionally answers in plain text. On the final turn, switching to"ANY"with a single allowed function name removes that ambiguity entirely.
Next Step
The single most useful action for tomorrow's work: take your current function-calling flow and add one more function that captures the final answer shape — Pattern 1. It is the smallest possible change that gives you structured responses, and migrating to Pattern 2 later is straightforward when complexity grows.
Thank you for reading. I hope it saves someone else the same afternoon of trial and error.