Gemini Output Not as Expected? Response Quality Troubleshooting FAQ

Getting started with Gemini often reveals quality issues in the generated output. Hallucinated facts, malformed JSON, outdated code examples, and truncated responses are common frustrations. The good news: most of these issues can be resolved through better prompt design and parameter tuning. This FAQ covers 7 of the most common output quality problems and practical solutions.

:::info This guide focuses on improving Gemini's output quality through prompt engineering and configuration. Each answer includes code examples and best practices. :::

Q1: Why does Gemini produce hallucinations (false information)?

Hallucinations are a well-known limitation of large language models, where the model confidently generates information that isn't factually accurate. Gemini is not immune to this behavior.

Common Hallucination Examples:

• Citing non-existent research papers or authors
• Providing inaccurate statistics or dates
• Describing API methods or functions that don't exist
• Misattributing quotes or facts to wrong sources

Root Causes:

1. Knowledge Cutoff Date

   • Gemini's training data ends at a specific date (April 2024 for current models)
   • Current events, recently published information, and recent updates are unknown

2. Ambiguous Prompts

   • Vague instructions encourage the model to "fill in the gaps"
   • The model will confidently guess when unsure

3. Information Not in Training Data

   • Niche topics or proprietary information not present in training data
   • Highly specialized technical domains with limited documentation

Mitigation Strategies:

1. Provide Specific, Detailed Instructions

Poor prompt:
"Tell me about Python"

Better prompt:
"Explain Python 3.12 context managers with implementation examples"

2. Explicitly Request Source Attribution

Prompt template:
"Answer the following question and cite your sources:
Question: What is the history of vaccine development?"

3. Supply Reference Documents (RAG)

Prompt template:
"Based ONLY on the following document, answer this question.
If information is not in the document, state: 'Not found in document.'

[Document content here]

Question: [Your question]"

4. Implement Confidence Indicators

Prompt template:
"Answer this question. If you're uncertain, prefix with 'I'm not certain, but...'
Question: [Your question]"

Python Implementation:

import google.generativeai as genai
 
def generate_with_guardrails(prompt):
    model = genai.GenerativeModel('gemini-pro')
 
    enhanced_prompt = f"""
    {prompt}
 
    IMPORTANT INSTRUCTIONS:
    1. Only state information you're confident about
    2. If uncertain, clearly state 'I'm not certain about this'
    3. Cite sources when possible
    4. Warn if information predates April 2024
    """
 
    response = model.generate_content(enhanced_prompt)
    return response.text
 
result = generate_with_guardrails("What are the latest AI trends?")
print(result)

:::tip While eliminating hallucinations completely is impossible, specific instructions and source attribution requests significantly reduce their frequency. :::

Q2: Why do I get different answers to the same question?

Asking Gemini the same question multiple times yields slightly different responses, which can be problematic for consistency-critical applications.

Understanding Temperature:

• Low (0.0-0.3): Deterministic, consistent responses
• Medium (0.5-0.7): Balanced, naturally varied responses
• High (0.8-1.0): Creative, highly varied responses

Why Responses Differ:

Default temperature is set around 0.7 for variety. This is intentional design—language models are probabilistic, meaning they select words based on probability distributions rather than fixed rules.

Getting Consistent Responses (Python):

import google.generativeai as genai
 
model = genai.GenerativeModel(
    'gemini-pro',
    generation_config=genai.types.GenerationConfig(
        temperature=0.0,  # Deterministic output
        top_p=0.95,
        top_k=40,
    )
)
 
# Run same prompt 3 times - responses will be nearly identical
for i in range(3):
    response = model.generate_content("What is the capital of Japan?")
    print(f"Answer {i+1}: {response.text}")

Temperature by Use Case:

Using REST API:

curl -X POST "https://generativelanguage.googleapis.com/v1/models/gemini-pro:generateContent?key=YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "contents": [{
      "parts": [{"text": "What is the capital of Japan?"}]
    }],
    "generationConfig": {
      "temperature": 0.0,
      "maxOutputTokens": 256
    }
  }'

:::info Low temperature for factual queries, high temperature for creative tasks—this fundamental principle guides proper model configuration. :::

Q3: Why is my JSON output malformed?

You request JSON format, yet Gemini returns invalid JSON with syntax errors, trailing text, or improperly escaped quotes.

Common Malformed JSON Issues:

• Extra text before or after the JSON object
• Unescaped quotes within string values
• Missing closing brackets or braces
• Incomplete structure

Example (Broken Output):

{
  "name": "Taro",
  "age": 30,
}
Some additional text here...

Solutions:

1. Explicit Format Instructions

Prompt template:
"Output ONLY valid JSON. Do not include any text before or after the JSON.
Ensure all quotes are properly escaped.

Data to convert: [Your data]"

2. Provide JSON Schema

Prompt template:
"Generate JSON following this schema:

{
  "type": "object",
  "properties": {
    "name": {"type": "string"},
    "age": {"type": "number"},
    "skills": {"type": "array", "items": {"type": "string"}}
  },
  "required": ["name", "age"]
}

Output data: [Your data]"

3. Python Implementation with Validation:

import google.generativeai as genai
import json
import re
 
def generate_json_safely(prompt, schema=None):
    model = genai.GenerativeModel('gemini-pro')
 
    enhanced_prompt = f"""
    {prompt}
 
    CRITICAL REQUIREMENTS:
    1. Output ONLY valid JSON
    2. No text before or after JSON
    3. All quotes must be properly escaped
    4. All brackets and braces must be closed
    """
 
    if schema:
        enhanced_prompt += f"\n\nJSON Schema:\n{json.dumps(schema, indent=2)}"
 
    response = model.generate_content(enhanced_prompt)
 
    try:
        # Extract JSON (remove any surrounding text)
        text = response.text
        start = text.find('{')
        end = text.rfind('}') + 1
 
        if start != -1 and end > start:
            json_str = text[start:end]
            return json.loads(json_str)
    except json.JSONDecodeError as e:
        print(f"JSON Parse Error: {e}")
        print(f"Raw response: {response.text}")
        return None
 
# Usage example
schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "number"}
    }
}
 
result = generate_json_safely("Person info: Name is Taro, age 30", schema)
if result:
    print(json.dumps(result, indent=2))

:::warning Always include JSON validation and error handling in production code—Gemini's format adherence is not guaranteed. :::

Q4: Why does generated code use outdated APIs?

Gemini sometimes generates code using old library versions, deprecated methods, or legacy patterns.

Example (TensorFlow 1.x Output):

# Outdated code Gemini might generate
import tensorflow as tf
sess = tf.Session()
result = sess.run(...)
 
# Correct modern TensorFlow 2.x
import tensorflow as tf
result = model(input_data)

Why This Happens:

• Training data contains more older documentation
• New API documentation may not be well represented in training data
• Older patterns are more common in training corpus

Solutions:

1. Specify Versions in Prompt

Prompt template:
"Generate Python 3.12 code using Flask 3.0.0.
Follow current best practices only. Do not use deprecated APIs.

Requirements: [Your requirements]"

2. Provide Reference Documentation (RAG)

def generate_code_with_docs(task, documentation):
    model = genai.GenerativeModel('gemini-pro')
 
    prompt = f"""
    Follow ONLY the provided documentation. Do not use any methods or patterns not shown here.
 
    DOCUMENTATION:
    {documentation}
 
    TASK:
    {task}
    """
 
    response = model.generate_content(prompt)
    return response.text
 
# Provide Flask 3.0 current docs
docs = """
Flask 3.0 route definition:
@app.route('/hello')
def hello():
    return jsonify({'message': 'Hello'})
"""
 
code = generate_code_with_docs("Create Flask API for dice", docs)

3. Validate Generated Code

import re
 
code = model.generate_content("...").text
 
# Detect deprecated patterns
deprecated_patterns = [
    r'tf\.Session\(',
    r'sess\.run\(',
    r'tf\.placeholder',
]
 
warnings = []
for pattern in deprecated_patterns:
    if re.search(pattern, code):
        warnings.append(f"Warning: Deprecated API detected: {pattern}")
 
if warnings:
    for w in warnings:
        print(w)
    # Request regeneration or manual review

:::tip Always verify generated code against official documentation and test with current library versions before deployment. :::

Q5: Why does my long response get cut off?

Generating longer content sometimes results in truncation due to token limits, leaving responses incomplete.

Example:

Prompt: "Write a 10,000-character detailed explanation of Python"
Result: Truncation halfway through (default limits are several thousand tokens)

Understanding Tokens:

• 1 token ≈ 4 characters (English)
• 1 token ≈ 2-3 characters (Japanese)
• max_output_tokens: Maximum tokens the model can generate

Solutions:

1. Increase max_output_tokens (Python)

import google.generativeai as genai
 
model = genai.GenerativeModel(
    'gemini-pro',
    generation_config=genai.types.GenerationConfig(
        max_output_tokens=4096,  # Increased from default
    )
)
 
response = model.generate_content("Explain Python in detail")
print(response.text)

2. Using REST API:

curl -X POST "https://generativelanguage.googleapis.com/v1/models/gemini-pro:generateContent?key=YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "contents": [{
      "parts": [{"text": "Detailed explanation needed"}]
    }],
    "generationConfig": {
      "maxOutputTokens": 4096
    }
  }'

3. Split Response into Sections

Prompt template:
"Explain Python in sections, generate one section at a time:
1. Core Concepts (1000 chars approx)
2. Language Features (1000 chars approx)
3. Ecosystem (1000 chars approx)

Start with section 1. After completing each section, I'll request the next."

Check Token Count:

import google.generativeai as genai
 
model = genai.GenerativeModel('gemini-pro')
 
prompt = "Your prompt here"
token_count = model.count_tokens(prompt)
print(f"Prompt tokens: {token_count.total_tokens}")

:::info Setting max_output_tokens defines the upper limit, but actual output length depends on the content and natural stopping points. :::

Q6: Why aren't System Instructions being applied?

You've configured System Instructions to guide model behavior, yet the output shows no difference.

What Are System Instructions:

System-level instructions that shape model behavior (e.g., "Always respond in formal tone", "Use bullet points").

Why They Don't Work:

1. User Prompt Overrides System Instruction

   • Direct user instructions can override system guidance

2. SDK Version Too Old

   • Older SDK versions may not support System Instructions

3. Conflicting Instructions

   • System Instructions and user prompt contradicting each other

Correct Implementation (Python):

import google.generativeai as genai
 
model = genai.GenerativeModel(
    'gemini-pro',
    system_instruction="""
    You are an experienced Python developer.
    Follow these rules:
    1. Always respond in formal tone
    2. Include working code examples
    3. End with 'Do you have any questions?'
    4. For errors, explain cause and solution
    """
)
 
response = model.generate_content("Tell me about Python list operations")
print(response.text)

REST API Configuration:

curl -X POST "https://generativelanguage.googleapis.com/v1/models/gemini-pro:generateContent?key=YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "system_instruction": {
      "parts": [{
        "text": "You are a technical support specialist. Answer politely and provide solutions."
      }]
    },
    "contents": [{
      "parts": [{"text": "Tell me about Python"}]
    }]
  }'

System Instruction Best Practices:

Good:
"You are a customer support representative.
Respond politely, explain clearly, and always offer next steps."

Avoid:
"Be helpful"

:::tip System Instructions apply to an entire model instance. Create separate instances if you need different behaviors for different conversations. :::

Q7: Why are images ignored in multimodal input?

You send both text and images, but Gemini analyzes only the text, ignoring image content.

Example (Image Ignored):

User: "Read the text in this image" [Image attached]
Gemini: "I cannot determine this from text alone..."

Causes:

1. Wrong Image Format

   • Supported: JPEG, PNG, GIF, WebP
   • Not supported: BMP, TIFF, and others

2. Image Too Small

   • Minimum width: 100 pixels recommended
   • For text extraction: 200 DPI or higher

3. Prompt Doesn't Reference Image

   • Instructions are text-only without image reference

Correct Implementation (Python):

import google.generativeai as genai
from PIL import Image
import base64
 
model = genai.GenerativeModel('gemini-pro-vision')
 
# Method 1: Direct file loading
response = model.generate_content([
    "What is in this image?",
    Image.open('/path/to/image.jpg')
])
 
# Method 2: Base64 encoding
with open('/path/to/image.jpg', 'rb') as f:
    image_data = base64.standard_b64encode(f.read()).decode('utf-8')
 
response = model.generate_content([
    {
        "text": "Extract all text from this image"
    },
    {
        "inline_data": {
            "mime_type": "image/jpeg",
            "data": image_data
        }
    }
])
 
print(response.text)

Processing Multiple Images:

import google.generativeai as genai
from PIL import Image
 
model = genai.GenerativeModel('gemini-pro-vision')
 
response = model.generate_content([
    "Compare these 3 images and explain their differences",
    Image.open('image1.jpg'),
    Image.open('image2.jpg'),
    Image.open('image3.jpg')
])
 
print(response.text)

Image Quality Guidelines:

• Minimum size: 100px × 100px
• File size: Under 20MB
• DPI: 200+ for text extraction
• Format: JPEG (efficiency) or PNG (quality)

:::warning Never upload images containing personal information, credentials, or sensitive data. :::

Looking back

Most Gemini output quality issues can be resolved through prompt engineering and proper configuration. Key takeaways:

• Hallucinations: Use specific instructions, request sources, implement RAG
• Inconsistent Responses: Adjust temperature (low for facts, high for creativity)
• Malformed JSON: Provide schema, validate parsing, include error handling
• Outdated Code: Specify versions, provide documentation, validate post-generation
• Truncated Output: Increase max_output_tokens or split response into sections
• System Instructions Not Working: Verify correct implementation, check SDK version
• Images Ignored: Use gemini-pro-vision, verify image format and size

Through prompt engineering and parameter tuning, you can consistently achieve high-quality output from Gemini.