Gemini 3.x Prompt Engineering Complete Masterclass — System Instructions, Few-shot, CoT, ReAct & Self-Evaluation Loops with Working Code

After working with the Gemini API for a while, most developers hit a wall they didn't expect. Prompts that worked perfectly suddenly produce worse results after a model update. Outputs that were consistent start varying unpredictably in production. The quality is "good enough" but never quite right.

The root cause is almost always the same: prompts that were written to "kind of work" rather than designed to work predictably. Gemini 3.x models have dramatically improved reasoning capabilities compared to 2.x — but that power cuts both ways. A well-designed prompt unlocks genuinely impressive outputs. A poorly designed one produces results that feel random.

This guide covers the major prompt engineering techniques for Gemini 3.1 Pro and 3.2, with working Python code and concrete before/after comparisons. I'll focus on what's changed with 3.x specifically, including some behaviors that aren't documented in the official guides.

What Changed in Gemini 3.x

Before diving into techniques, it's worth understanding what's actually different about 3.x compared to 2.x models.

Change 1: System Instructions now shape the reasoning process, not just the style

In 2.x, System Instructions were primarily a style guide — they'd nudge tone and format. In 3.x, they appear to influence the model's internal reasoning process. Experiments show that structured System Instructions with explicit constraints produce significantly more consistent outputs.

Change 2: Few-shot examples can include reasoning chains

Traditional Few-shot Learning showed input/output pairs. With 3.x, including reasoning traces in your examples causes the model to mirror that reasoning style. This dramatically improves consistency on complex tasks.

Change 3: Self-evaluation loops are now practical

In 2.x, asking the model to critique its own output often produced superficial feedback. Gemini 3.x (especially 3.1 Pro) produces substantive, actionable critiques that actually improve output quality in iteration.

Change 4: Thinking Budget is a controllable variable

This is the biggest structural change. You can now tell the model how much internal computation to use before responding — enabling a quality/cost/latency tradeoff that didn't exist before.

Environment Setup

# pip install google-genai
 
import os
from google import genai
from google.genai import types
 
client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])
MODEL = "gemini-3.1-pro"
 
def generate(prompt: str, system: str = "", thinking_budget: int = 0) -> str:
    """Simple text generation helper."""
    config = types.GenerateContentConfig(
        system_instruction=system if system else None,
        thinking_config=types.ThinkingConfig(
            thinking_budget=thinking_budget
        ) if thinking_budget > 0 else None,
    )
    response = client.models.generate_content(
        model=MODEL,
        contents=prompt,
        config=config,
    )
    return response.text

System Instructions: Production Design Patterns

System Instructions are the reasoning framework you hand to the model before it sees any user input. In Gemini 3.x, the quality of this framework determines the ceiling of your output quality.

Before: The Vague Instruction Problem

# ❌ What most developers write initially
bad_system = "You are a helpful AI assistant. Answer the user's questions."
 
response = generate(
    prompt="How do I read a file in Python?",
    system=bad_system
)
# → You'll get a generic, padded explanation with too much explanation
# and not enough useful code

After: Structured System Instructions

# ✅ Role + constraints + output format + forbidden patterns
good_system = """
# Role
You are a technical writer specializing in Python, targeting indie developers
who need production-ready code, not textbook examples.
 
# Output Principles
- Only show code that you'd stake your reputation on working
- Comments must explain WHY, not WHAT
- Always include error handling
- Prioritize practical over theoretical
 
# Forbidden Patterns
- No greeting phrases like "Great question!"
- No hedging language like "you might want to" or "could potentially"
- No unnecessary preamble or postscript
- No "As an AI language model" disclaimers
 
# Output Format
Code block → one-line context → gotchas (only if they exist)
"""
 
response = generate(
    prompt="How do I read a file in Python?",
    system=good_system
)
# → Concise, production-ready code with a comment explaining why
# open() is preferred over pathlib for this use case — useful output

The System Instructions Template

Here's the template I use in production:

SYSTEM_TEMPLATE = """
# Persona
{persona}
 
# Task Definition
{task_definition}
 
# Input Format
{input_format}
 
# Output Constraints
{output_constraints}
 
# Forbidden Patterns (responses matching these are considered failures)
{forbidden_patterns}
 
# Output Example (target this quality level)
{output_example}
"""
 
def build_system(
    persona: str,
    task_definition: str,
    input_format: str = "Natural language from the user",
    output_constraints: str = "",
    forbidden_patterns: str = "",
    output_example: str = ""
) -> str:
    return SYSTEM_TEMPLATE.format(
        persona=persona,
        task_definition=task_definition,
        input_format=input_format,
        output_constraints=output_constraints,
        forbidden_patterns=forbidden_patterns,
        output_example=output_example,
    ).strip()

Formalizing your System Instructions like this makes them easier to version, test, and hand off to collaborators.

Few-shot Learning: Selecting and Positioning Examples

Few-shot Learning in Gemini 3.x is more nuanced than showing input/output pairs. The model learns from the reasoning style embedded in your examples, not just the format.

Principle 1: Include Edge Cases

# ✅ Good few-shot example selection: cover the edges
few_shot_examples = [
    # Typical case
    {
        "input": "What's the weather in Tokyo?",
        "output": "I don't have access to real-time data, so I can't give you current conditions. Weather services like weather.com or the Japan Meteorological Agency (jma.go.jp) would have accurate forecasts."
    },
    # Edge case: ambiguous request
    {
        "input": "Weather?",
        "output": "Which location are you asking about? Once you share a city or region, I can point you to the right forecasting resource."
    },
    # Edge case: multi-location comparison
    {
        "input": "Compare Tokyo and Osaka weather for the next 3 days",
        "output": "I don't have access to live weather feeds. For a side-by-side forecast, weather.com's comparison feature or Weather Underground work well for this."
    }
]

Principle 2: Embed Reasoning Traces (Gemini 3.x Specific)

# Including reasoning in examples causes Gemini 3.x to mirror
# that reasoning style — this is more effective than plain I/O pairs
 
few_shot_with_reasoning = """
Example 1:
Input: "Analyze the sentiment: 'Today was absolutely terrible'"
Reasoning: Strong negative word "absolutely terrible." The phrasing feels more like
disappointment/resignation than anger — diary-like tone, not confrontational.
Output: {"sentiment": "negative", "intensity": 0.85, "dominant_emotion": "disappointment", "confidence": 0.88}
 
Example 2:
Input: "Analyze the sentiment: 'It was okay I guess'"
Reasoning: "Okay" and "I guess" both signal neutral with mild dissatisfaction.
Neither positive nor negative, low intensity.
Output: {"sentiment": "neutral", "intensity": 0.2, "dominant_emotion": "indifference", "confidence": 0.80}
 
Now analyze: "Finally done, and honestly it went better than I expected"
"""
 
response = generate(few_shot_with_reasoning)
# → The model will produce a reasoning trace before its JSON output,
# matching the pattern shown in examples

Positioning Strategy

Through testing, I've found that example placement matters more than most guides acknowledge:

def build_few_shot_prompt(
    task_description: str,
    examples: list[dict],
    actual_input: str,
    placement: str = "before"
) -> str:
    """
    placement="before": All examples before the actual input (works well for classification)
    placement="after": Task description first, then examples (works well for complex tasks)
    placement="mixed": One example inline with the task description (compact prompts)
    """
    examples_text = "\n\n".join([
        f"Input: {ex['input']}\nOutput: {ex['output']}"
        for ex in examples
    ])
    
    if placement == "before":
        return f"{examples_text}\n\nInput: {actual_input}\nOutput:"
    elif placement == "after":
        return f"{task_description}\n\n{examples_text}\n\nInput: {actual_input}\nOutput:"
    else:  # mixed
        first_example = examples[0]
        return (
            f"{task_description}\n\n"
            f"Example: Input '{first_example['input']}' → Output '{first_example['output']}'\n\n"
            f"Input: {actual_input}\nOutput:"
        )

For complex tasks involving JSON output or multi-step reasoning, placement="after" consistently performs better in my testing. For binary classification and sentiment analysis, placement="before" is more stable.

Chain-of-Thought + Thinking Budget Integration

This is where Gemini 3.x diverges most from earlier models. The Thinking Budget parameter lets you dial in how much internal computation the model uses before responding.

def generate_with_cot(
    question: str,
    thinking_budget: int = 8192,
    explicit_cot: bool = True
) -> dict:
    """
    Chain-of-Thought generation with optional Thinking Budget.
    thinking_budget=0: No internal reasoning (fastest, cheapest)
    thinking_budget=32768: Maximum reasoning depth (slowest, most expensive)
    """
    if explicit_cot:
        prompt = f"""
{question}
 
Before answering, work through these steps:
1. What's the core question really asking?
2. What information or assumptions are needed?
3. If there are multiple approaches, what are the tradeoffs?
4. Why is one approach better than the others?
5. Final answer
 
Wrap each step in <thinking> tags, then give your final answer in <answer> tags.
"""
    else:
        prompt = question
    
    config = types.GenerateContentConfig(
        thinking_config=types.ThinkingConfig(thinking_budget=thinking_budget)
    )
    
    response = client.models.generate_content(
        model=MODEL,
        contents=prompt,
        config=config,
    )
    
    text = response.text
    thinking_part = ""
    answer_part = text
    
    if "<thinking>" in text and "</thinking>" in text:
        start = text.index("<thinking>") + len("<thinking>")
        end = text.index("</thinking>")
        thinking_part = text[start:end].strip()
        
    if "<answer>" in text and "</answer>" in text:
        start = text.index("<answer>") + len("<answer>")
        end = text.index("</answer>")
        answer_part = text[start:end].strip()
    
    return {
        "thinking": thinking_part,
        "answer": answer_part,
        "usage": response.usage_metadata.dict() if response.usage_metadata else {}
    }
 
# Usage
result = generate_with_cot(
    question="Should I price my SaaS at $10/month or $20/month? "
             "Target is indie developers, competitors range from $8-15/month.",
    thinking_budget=8192
)
 
print("Reasoning trace:", result["thinking"][:300], "...")
print("\nAnswer:", result["answer"])
# → With Thinking Budget, the model works through price elasticity,
# market positioning, and conversion rate implications — not just
# "pick the middle price" advice

Calibrating Thinking Budget

# Practical reference guide for budget selection
THINKING_BUDGET_GUIDE = {
    "Simple Q&A": 0,                    # Free — fastest
    "Structured data extraction": 1024,
    "Code generation (under 100 lines)": 4096,
    "Complex business logic decisions": 8192,
    "Multi-step reasoning / strategy": 16384,
    "Maximum accuracy required": 32768, # Most expensive
}
 
def auto_thinking_budget(prompt: str) -> int:
    """Estimate appropriate budget from prompt characteristics."""
    has_multiple_constraints = (
        prompt.count(" and ") + prompt.count(" but ") + prompt.count(" however ") > 2
    )
    has_calculation = any(kw in prompt.lower() for kw in ["calculate", "analyze", "compare", "evaluate"])
    is_long = len(prompt) > 500
    
    score = sum([has_multiple_constraints, has_calculation, is_long])
    return {0: 0, 1: 2048, 2: 8192, 3: 16384}.get(score, 4096)

The practical recommendation: start with thinking_budget=0 for everything, then only increase it for specific tasks where you observe quality gaps. Blanket application of high budgets is one of the fastest ways to generate an unexpected cost spike.

ReAct Pattern: Reasoning + Acting + Observation Loops

ReAct combines reasoning with tool use in an iterative loop. Unlike simple Function Calling where you define the tool invocation upfront, ReAct lets the model decide which tools to use and in what order.

import json
from typing import Callable
 
def react_agent(
    task: str,
    tools: dict[str, Callable],
    max_steps: int = 5,
    system: str = ""
) -> str:
    """
    Minimal ReAct agent implementation.
    tools: {"tool_name": callable_function} mapping
    Each function's docstring is used as the tool description.
    """
    tools_description = "\n".join([
        f"- {name}: {func.__doc__ or 'No description'}"
        for name, func in tools.items()
    ])
    
    react_system = f"""
{system}
 
# Available Tools
{tools_description}
 
# ReAct Process
At each step, output in this exact format:
 
Thought: [Analyze current situation and decide next action]
Action: [Tool name]
Action Input: [JSON input for the tool]
Observation: [After receiving tool output, describe what you learned]
... (repeat as needed)
Final Answer: [Your final response to the original task]
"""
    
    history = [{"role": "user", "content": task}]
    
    for step in range(max_steps):
        response = client.models.generate_content(
            model=MODEL,
            contents=[
                {"role": m["role"], "parts": [{"text": m["content"]}]}
                for m in history
            ],
            config=types.GenerateContentConfig(
                system_instruction=react_system,
                thinking_config=types.ThinkingConfig(thinking_budget=4096)
            )
        )
        
        text = response.text
        history.append({"role": "model", "content": text})
        
        if "Final Answer:" in text:
            return text.split("Final Answer:")[-1].strip()
        
        if "Action:" in text and "Action Input:" in text:
            try:
                action = text.split("Action:")[1].split("\n")[0].strip()
                raw_input = text.split("Action Input:")[1].split("\n")[0].strip()
                tool_input = json.loads(raw_input)
                
                if action in tools:
                    result = tools[action](**tool_input)
                    observation = f"Observation: {result}"
                else:
                    observation = f"Observation: Error — tool '{action}' does not exist"
                
                history.append({"role": "user", "content": observation})
            except (json.JSONDecodeError, KeyError, TypeError) as e:
                history.append({"role": "user", "content": f"Observation: Error — {str(e)}"})
    
    return f"Reached max steps. Last response: {history[-2]['content']}"
 
# Example usage
def search_web(query: str) -> str:
    """Search the web and return relevant information."""
    return f"Search results for '{query}': [Sample data for demonstration]"
 
def calculate(expression: str) -> str:
    """Evaluate a mathematical expression and return the result."""
    try:
        result = eval(expression, {"__builtins__": {}})
        return str(result)
    except Exception as e:
        return f"Calculation error: {str(e)}"
 
tools = {"search_web": search_web, "calculate": calculate}
 
result = react_agent(
    task="If I need $1M ARR from a $20/month SaaS, how many paying customers do I need? "
         "Also research the typical conversion rate from free trial to paid for B2B SaaS tools.",
    tools=tools,
    max_steps=5
)
print(result)

The key insight with ReAct in Gemini 3.x: the model handles ambiguity better than previous versions. When a task requires multiple tools in a non-obvious sequence, 3.x models tend to figure out the right order without explicit guidance.

Critic-Refiner: Self-Evaluation Loops

The self-evaluation capability is where Gemini 3.x shows the most improvement over 2.x. Here's a practical implementation:

def critic_refiner(
    original_prompt: str,
    draft_output: str,
    evaluation_criteria: list[str],
    max_iterations: int = 3
) -> dict:
    """
    Critic-Refiner loop.
    Scores the output against criteria, then refines until thresholds are met.
    Returns: {final_output, iterations, total_iterations}
    """
    current_output = draft_output
    iterations = []
    
    for i in range(max_iterations):
        # Step 1: Critique
        critic_prompt = f"""
Evaluate the following output against the given criteria.
 
# Original Task
{original_prompt}
 
# Current Output
{current_output}
 
# Evaluation Criteria
{chr(10).join([f"- {c}" for c in evaluation_criteria])}
 
Score each criterion 1-10 with specific feedback.
If all criteria score 8+, verdict is "APPROVED". Otherwise "NEEDS_IMPROVEMENT".
 
Return valid JSON in this exact format:
{{
  "scores": {{"criterion_name": score_int}},
  "average_score": float,
  "verdict": "APPROVED" or "NEEDS_IMPROVEMENT",
  "improvements": ["specific improvement 1", "specific improvement 2"]
}}
"""
        critic_response = client.models.generate_content(
            model=MODEL,
            contents=critic_prompt,
            config=types.GenerateContentConfig(
                response_mime_type="application/json",
                thinking_config=types.ThinkingConfig(thinking_budget=4096)
            )
        )
        
        try:
            evaluation = json.loads(critic_response.text)
        except json.JSONDecodeError:
            break
        
        iterations.append({
            "iteration": i + 1,
            "output": current_output,
            "evaluation": evaluation
        })
        
        if evaluation.get("verdict") == "APPROVED":
            break
        
        # Step 2: Refine
        improvements = evaluation.get("improvements", [])
        if not improvements:
            break
            
        refiner_prompt = f"""
Improve the following output based on the feedback below.
Return only the improved output — no explanations or meta-commentary.
 
# Original Task
{original_prompt}
 
# Current Output
{current_output}
 
# Required Improvements
{chr(10).join([f"- {imp}" for imp in improvements])}
"""
        
        refiner_response = client.models.generate_content(
            model=MODEL,
            contents=refiner_prompt,
            config=types.GenerateContentConfig(
                thinking_config=types.ThinkingConfig(thinking_budget=8192)
            )
        )
        current_output = refiner_response.text
    
    return {
        "final_output": current_output,
        "iterations": iterations,
        "total_iterations": len(iterations)
    }

In practice, one iteration is usually enough for most quality improvements. Running two iterations is worthwhile for high-stakes content like customer-facing copy or technical documentation. Three or more iterations rarely produce meaningful gains relative to cost.

Common Mistakes and Pitfalls

Pitfall 1: System Instructions vs. User Prompt Role Confusion

# ❌ Putting session-specific data in System Instructions
bad_system = """
You're an assistant for Tanaka Taro (CTO at Example Corp).
Current project: E-commerce redesign (deadline: July 2026)
Team: 5 people
Stack: Next.js, PostgreSQL, Cloudflare Workers
"""
# Problem: If context changes between users, you need to rebuild System Instructions,
# which breaks context caching and increases costs.
 
# ✅ System Instructions for stable persona, User Prompt for session context
good_system = "You are a project management advisor. Base your responses on the context provided."
user_prompt = """
Context:
- Person: Tanaka Taro (CTO)
- Project: E-commerce redesign (deadline: July 2026)
- Team: 5 people, Stack: Next.js, PostgreSQL, Cloudflare Workers
 
Question: How should we prioritize next sprint?
"""

Pitfall 2: Applying High Thinking Budget Universally

# ❌ This runs up costs fast
always_max_config = types.GenerateContentConfig(
    thinking_config=types.ThinkingConfig(thinking_budget=32768)
)
 
# ✅ Dynamic budget allocation
def get_appropriate_budget(prompt: str) -> int:
    """Simple heuristic for budget estimation."""
    has_multiple_constraints = (
        prompt.count("and also") + prompt.count("but also") + prompt.count("additionally") > 2
    )
    has_analysis = any(kw in prompt.lower() for kw in ["analyze", "calculate", "compare", "evaluate"])
    is_long = len(prompt) > 500
    
    score = sum([has_multiple_constraints, has_analysis, is_long])
    return {0: 0, 1: 2048, 2: 8192, 3: 16384}.get(score, 4096)

Pitfall 3: Contradictory Few-shot Examples

# ❌ Mixed styles confuse the model
contradictory = [
    {"input": "Hello", "output": "Yes, how may I assist?"},    # Formal
    {"input": "Hey", "output": "Hey! What's up?"},             # Casual
    {"input": "Good morning", "output": "How can I help?"},    # Neutral
]
# The model doesn't know which register to use for new inputs
 
# ✅ Pick one style and stick with it
consistent_formal = [
    {"input": "Hello", "output": "Good day. How may I assist you?"},
    {"input": "Hey", "output": "Hello. How can I help you today?"},
    {"input": "Good morning", "output": "Good morning. What can I do for you?"},
]

Prompt Version Management for Production

Prompts are production artifacts and should be version-controlled like code:

import hashlib
from datetime import datetime
 
class PromptVersion:
    """Lightweight prompt versioning for production tracking."""
    
    def __init__(self, system: str, examples: list[dict] = None, template: str = ""):
        self.system = system
        self.examples = examples or []
        self.template = template
        self.created_at = datetime.utcnow().isoformat()
        self.version_hash = self._compute_hash()
    
    def _compute_hash(self) -> str:
        content = f"{self.system}|{str(self.examples)}|{self.template}"
        return hashlib.sha256(content.encode()).hexdigest()[:8]
    
    def render(self, **kwargs) -> str:
        return self.template.format(**kwargs)
    
    def to_dict(self) -> dict:
        return {
            "version_hash": self.version_hash,
            "created_at": self.created_at,
            "system": self.system,
            "examples_count": len(self.examples),
        }
 
# Usage
v1 = PromptVersion(system="You are a code reviewer.", template="Review this code:\n{code}")
v2 = PromptVersion(
    system="You are a senior engineer reviewing code for security, performance, and readability.",
    template="Review this Python code:\n```python\n{code}\n```"
)
 
print(f"v1 hash: {v1.version_hash}")  # a1b2c3d4
print(f"v2 hash: {v2.version_hash}")  # e5f6g7h8
# When you log evaluation results alongside version_hash, you can
# track which prompt version was responsible for which output quality

Putting It All Together: Full Pipeline

class PromptPipeline:
    """
    Production prompt pipeline combining Few-shot + CoT + Critic-Refiner.
    """
    
    def __init__(
        self,
        system: str,
        examples: list[dict],
        evaluation_criteria: list[str],
        thinking_budget: int = 4096
    ):
        self.system = system
        self.examples = examples
        self.evaluation_criteria = evaluation_criteria
        self.thinking_budget = thinking_budget
    
    def _build_prompt(self, user_input: str) -> str:
        examples_text = "\n\n".join([
            f"Input: {ex['input']}\nOutput: {ex['output']}"
            for ex in self.examples
        ])
        return f"{examples_text}\n\nInput: {user_input}\nOutput:"
    
    def run(self, user_input: str, auto_refine: bool = True) -> dict:
        prompt = self._build_prompt(user_input)
        
        # Step 1: Initial generation
        config = types.GenerateContentConfig(
            system_instruction=self.system,
            thinking_config=types.ThinkingConfig(thinking_budget=self.thinking_budget)
        )
        response = client.models.generate_content(
            model=MODEL, contents=prompt, config=config
        )
        draft = response.text
        
        if not auto_refine:
            return {"output": draft, "refined": False}
        
        # Step 2: Critic-Refiner loop
        result = critic_refiner(
            original_prompt=user_input,
            draft_output=draft,
            evaluation_criteria=self.evaluation_criteria,
            max_iterations=2
        )
        
        return {
            "output": result["final_output"],
            "refined": True,
            "iterations": result["total_iterations"],
        }
 
# Usage
pipeline = PromptPipeline(
    system="You are a SaaS product strategy advisor.",
    examples=[
        {
            "input": "How do I find out why users are churning?",
            "output": "Add an exit survey to your cancellation flow: three radio buttons (price, missing feature, switching to competitor) plus optional free-text. First-month churn tends to be highest, so treat this as an onboarding quality metric too."
        }
    ],
    evaluation_criteria=[
        "Contains at least one concrete, actionable recommendation",
        "References a measurable outcome (metric, KPI, or timeframe)",
        "Includes a potential risk or caveat",
    ],
    thinking_budget=8192
)
 
result = pipeline.run("My monthly churn rate is over 5%. What should I focus on first?")
print(result["output"])

Advanced Pattern: Prompt Chaining for Multi-Step Tasks

For tasks too complex for a single prompt, prompt chaining breaks the work into stages where each stage's output becomes the next stage's input. This sounds obvious, but the design choices in how you chain prompts matter significantly.

from dataclasses import dataclass
 
@dataclass
class ChainStep:
    """A single step in a prompt chain."""
    name: str
    system: str
    template: str  # Uses {previous_output} to reference prior step
    thinking_budget: int = 0
    requires_json: bool = False
 
class PromptChain:
    """
    Multi-step prompt chain where each step builds on the previous.
    Useful for: document analysis → summary → action items → email draft
    """
    
    def __init__(self, steps: list[ChainStep]):
        self.steps = steps
    
    def run(self, initial_input: str, verbose: bool = False) -> dict:
        results = {}
        current_output = initial_input
        
        for step in self.steps:
            prompt = step.template.format(
                previous_output=current_output,
                initial_input=initial_input
            )
            
            config = types.GenerateContentConfig(
                system_instruction=step.system if step.system else None,
                thinking_config=types.ThinkingConfig(
                    thinking_budget=step.thinking_budget
                ) if step.thinking_budget > 0 else None,
                response_mime_type="application/json" if step.requires_json else None,
            )
            
            response = client.models.generate_content(
                model=MODEL,
                contents=prompt,
                config=config,
            )
            
            current_output = response.text
            results[step.name] = current_output
            
            if verbose:
                print(f"✓ Step '{step.name}' completed ({len(current_output)} chars)")
        
        return results
 
# Example: Meeting transcript → structured action items → follow-up email
meeting_chain = PromptChain([
    ChainStep(
        name="extract_key_points",
        system="You extract structured information from meeting transcripts. Be thorough and precise.",
        template="""
Extract all key decisions, action items, and open questions from this meeting transcript.
 
Transcript:
{initial_input}
 
Return JSON: {{"decisions": [], "action_items": [{{"owner": "", "task": "", "deadline": ""}}], "open_questions": []}}
""",
        thinking_budget=4096,
        requires_json=True,
    ),
    ChainStep(
        name="draft_followup_email",
        system="You write clear, professional follow-up emails that people actually read.",
        template="""
Write a concise follow-up email based on this meeting summary.
Keep it under 150 words. No fluffy opener or closer.
 
Meeting summary:
{previous_output}
""",
        thinking_budget=0,
    ),
])
 
transcript = """
[10:02] Alice: We need to decide on the database — PostgreSQL or MongoDB.
[10:05] Bob: I lean PostgreSQL, the data is relational. Agreed. Let's go with PostgreSQL.
[10:07] Alice: Bob, can you set up the schema by Friday?
[10:09] Bob: Sure. I'll also design with read replicas in mind.
[10:11] Alice: I'll send design specs by Wednesday and follow up with DevOps about staging.
"""
 
results = meeting_chain.run(transcript, verbose=True)
print(results["draft_followup_email"])

The practical rule: if later stages need to behave differently based on what earlier stages found, use chaining. If it's a complex-but-uniform task, a single structured prompt is faster and cheaper.

Temperature and Sampling Parameters

Thinking Budget captures most developer attention, but sampling parameters affect consistency in ways that are easy to overlook.

# Task-specific sampling configs I use in production
SAMPLING_CONFIGS = {
    # Data extraction, classification, structured output
    "factual": {"temperature": 0.1, "top_p": 0.9, "top_k": 40},
    
    # Technical documentation, code review
    "technical": {"temperature": 0.3, "top_p": 0.92, "top_k": 50},
    
    # Default for most tasks
    "balanced": {"temperature": 1.0, "top_p": 0.95, "top_k": 64},
    
    # Marketing copy, creative writing
    "creative": {"temperature": 1.5, "top_p": 0.99, "top_k": 100},
}
 
def generate_with_sampling(prompt: str, system: str = "", task_type: str = "balanced") -> str:
    config_params = SAMPLING_CONFIGS[task_type]
    config = types.GenerateContentConfig(
        system_instruction=system if system else None,
        temperature=config_params["temperature"],
        top_p=config_params["top_p"],
        top_k=config_params["top_k"],
    )
    response = client.models.generate_content(model=MODEL, contents=prompt, config=config)
    return response.text

One Gemini 3.x specific behavior: when Thinking Budget is active, temperature has less impact on output than you'd expect. The reasoning process stabilizes the output. This means you can use slightly higher temperatures for creative tasks without losing coherence — a genuine improvement over 2.x behavior.

For classification, data extraction, and code generation, drop temperature to 0.1-0.3. The consistency improvement is immediately visible in production logs.

A Minimal Evaluation Framework

Measuring prompt quality doesn't require a separate evaluation system. Gemini itself can score outputs:

import statistics
 
class PromptEvaluator:
    """Lightweight prompt A/B testing using Gemini as judge."""
    
    def __init__(self, criteria: list[str]):
        self.criteria = criteria
    
    def score(self, task: str, output: str) -> float:
        """Score output against criteria. Returns 0.0-10.0."""
        prompt = f"""
Rate this output 1-10 against each criterion. Return JSON only.
 
Task: {task}
Output: {output}
Criteria: {self.criteria}
 
JSON: {{"scores": {{"criterion": int}}, "overall": float}}
"""
        r = client.models.generate_content(
            model=MODEL, contents=prompt,
            config=types.GenerateContentConfig(response_mime_type="application/json")
        )
        try:
            return json.loads(r.text).get("overall", 0.0)
        except Exception:
            return 0.0
    
    def compare_versions(
        self,
        v1: PromptVersion,
        v2: PromptVersion,
        test_inputs: list[str],
        n_samples: int = 3
    ) -> dict:
        """Compare two prompt versions across test inputs."""
        v1_scores, v2_scores = [], []
        
        for inp in test_inputs:
            for _ in range(n_samples):
                o1 = generate(v1.render(input=inp), system=v1.system)
                o2 = generate(v2.render(input=inp), system=v2.system)
                v1_scores.append(self.score(inp, o1))
                v2_scores.append(self.score(inp, o2))
        
        return {
            "v1": {"hash": v1.version_hash, "avg": statistics.mean(v1_scores), "std": statistics.stdev(v1_scores) if len(v1_scores) > 1 else 0},
            "v2": {"hash": v2.version_hash, "avg": statistics.mean(v2_scores), "std": statistics.stdev(v2_scores) if len(v2_scores) > 1 else 0},
            "winner": "v1" if statistics.mean(v1_scores) > statistics.mean(v2_scores) else "v2"
        }
 
evaluator = PromptEvaluator(criteria=[
    "Specific and actionable",
    "Covers risks and tradeoffs",
    "Appropriate length",
])

Track standard deviation alongside average score. A prompt with high average but high variance will produce inconsistent user experiences. Low variance is a sign of a well-constrained prompt.

Where to Go Next

The highest-leverage change you can make right now is restructuring your System Instructions. Add a "Forbidden Patterns" section to whatever you're currently using and test it against ten real inputs. The consistency improvement is usually immediate.

From there, the learning path I'd recommend:

Start with Gemini API System Instructions and Prompt Design for foundational patterns
Then Gemini 3 Deep Think Production: Advanced Reasoning Patterns for Thinking Budget integration
Finally RSFC Structured Prompt Complete Guide for Gemini as an alternative framework worth knowing

The techniques in this guide compound. System Instructions set the baseline, Few-shot raises the ceiling, CoT unlocks complex reasoning, and the Critic-Refiner loop catches the edge cases. Implementing them together is what separates "good enough" from "production-grade."