I have been running a few wallpaper apps as an indie developer for years, and recently I built a pipeline on top of gemini-2.5-flash-image — the model commonly called Nano Banana — to derive color variations from existing wallpapers. It worked beautifully in the prompt playground. As soon as I scaled the same calls into a batch, the model started returning completely different artwork, ignoring the reference I had attached. Asking for "just a bluer version of this" returned a brand new composition.
Plenty of developers seem to hit the same wall, so here is the triage order I now use whenever the reference image looks ignored. The fixes are ordered by how often they turn out to be the actual cause in production.
Pin down the symptom first
Before changing any code, classify what you are seeing. The cause is different for each of these, and treating them the same wastes hours.
- A: A completely unrelated image is returned (no subject, composition, or palette survives)
- B: Only text comes back ("I cannot generate that image…")
- C: The reference faintly survives, but the edit instruction ("only change the color") is ignored
A and B are almost always request-construction bugs. C is usually about prompt design and the order of the parts array.
Cause 1: response_modalities is missing IMAGE
This is the first thing to check. When you omit response_modalities, the SDK can default to TEXT-only, which puts the model in "describe what I would have edited" mode rather than the editing mode you want. That single setting accounts for most of symptom B.
from google import genai
from google.genai import types
client = genai.Client(api_key="YOUR_GEMINI_API_KEY")
with open("base_wallpaper.png", "rb") as f:
base_image_bytes = f.read()
response = client.models.generate_content(
model="gemini-2.5-flash-image",
contents=[
types.Part.from_bytes(data=base_image_bytes, mime_type="image/png"),
"Take this image as the base. Recolor the entire palette to a night-time blue. Keep composition and subject unchanged.",
],
config=types.GenerateContentConfig(
response_modalities=["IMAGE"], # ← if this is missing you may get text only
),
)["IMAGE"] alone or ["IMAGE", "TEXT"] together are both fine. ["TEXT"] alone, even by accident, will produce no image part.
Cause 2: parts order and MIME type
Nano Banana is more sensitive to ordering than the documentation suggests. In production, putting the image part first and the instruction second measurably improves edit consistency. Flipping the order tends to push the model toward treating the text as the primary request and the image as a loose stylistic reference, which feeds symptoms A and C.
Always set the MIME type to match the actual bytes. If you label a JPEG as image/png the decode step can silently drop the request, which feels like "the first call mysteriously ignores my image."
import imghdr
mime_map = {"png": "image/png", "jpeg": "image/jpeg", "jpg": "image/jpeg", "webp": "image/webp"}
fmt = imghdr.what(None, h=base_image_bytes)
mime = mime_map.get(fmt or "", "image/png")Cause 3: the image is too big or too small
This is the trap that cost me the most debugging time on my own pipeline. Sending a 4K wallpaper (3840×2160) returned no error but produced a "thumbnail-grade" alternative image. Going the other way and sending anything 128×128 or smaller made the model behave as if no image had been attached at all.
The sweet spot I landed on is a longest side between 1024 and 1568 pixels. Larger inputs are internally downscaled in a way that loses the fine details the edit relies on. Pre-resizing on the client side made the output dramatically more consistent across my wallpaper batch jobs.
from PIL import Image
import io
def normalize_for_nano_banana(path: str, max_side: int = 1568) -> bytes:
img = Image.open(path).convert("RGB")
w, h = img.size
scale = min(max_side / max(w, h), 1.0)
if scale < 1.0:
img = img.resize((int(w * scale), int(h * scale)), Image.LANCZOS)
buf = io.BytesIO()
img.save(buf, format="PNG", optimize=True)
return buf.getvalue()The convert("RGB") call is intentional. PNGs with an alpha channel can lead the model to repaint the transparent regions with a new background, which effectively rewrites the composition. If you do not need transparency, drop it.
Cause 4: the prompt conflates "generate" and "edit"
The same model performs both image generation and image editing. When the prompt is ambiguous, the request internally drifts toward the "new generation" branch, and the reference image is demoted to a style hint. Symptoms A and C live here.
Here are real prompts from my wallpaper pipeline and how much they shifted the hit rate.
- ❌ "Make a night version of it" — drifts into generation, returns something unrelated
- ❌ "Same vibe but at night" — "vibe" is too vague, composition shifts
- ✅ "Using the attached image as the base, keep the composition and subject identical, change only the palette to a night-time blue. Do not modify outlines, layout, or pose."
The trick is to separate what to preserve (composition, subject, pose) from what to change (palette only) and state the "do not change" clause first. That keeps the model anchored in editing mode.
Cause 5: chat sessions drop the reference after turn 2
When you iterate on an image across multiple turns inside a single chats.create() session, turn 2 and onward sometimes lose track of the original image. Older image parts get squeezed by token compression and the model can no longer "see" them.
In my pipeline I gave up on long chat sessions for editing flows. Instead, every turn re-attaches the latest output image as fresh input. Treating the workflow as stateless ("the previous result is the next input") produces far more predictable edits.
def edit_iteratively(client, prev_image_bytes: bytes, instruction: str) -> bytes:
response = client.models.generate_content(
model="gemini-2.5-flash-image",
contents=[
types.Part.from_bytes(data=prev_image_bytes, mime_type="image/png"),
instruction,
],
config=types.GenerateContentConfig(response_modalities=["IMAGE"]),
)
for part in response.candidates[0].content.parts:
if part.inline_data and part.inline_data.data:
return part.inline_data.data
raise RuntimeError("No image part in the response. Check prompt and response_modalities.")Always assert that the response actually contains an image
Nano Banana mixes image parts into the response parts array alongside any text. If you reach for response.text and call it a day, you may silently save None or a polite refusal, and then your downstream pipeline ships broken wallpapers without anyone noticing.
I run every response through an extractor and treat "no image part" as an immediate retry signal:
def extract_image_or_raise(response) -> bytes:
parts = response.candidates[0].content.parts if response.candidates else []
for p in parts:
if getattr(p, "inline_data", None) and p.inline_data.data:
return p.inline_data.data
text = " | ".join(getattr(p, "text", "") or "" for p in parts)
raise RuntimeError(f"No image in response. Text was: {text[:200]}")Logging the text alongside the failure makes it obvious whether you hit a safety filter or your prompt drifted into generation mode. That distinction speeds up triage by an order of magnitude.
My triage order, summarized
When this kind of bug appears inside a pipeline that feeds my wallpaper apps, debugging time compounds quickly, so I always check in this fixed order:
- Does
response_modalitiesincludeIMAGE? (about 80% of symptom B) - Are parts ordered image-then-text?
- Does the MIME type match the actual bytes?
- Is the longest side between 1024 and 1568 pixels?
- Does the prompt name what to preserve and what to change explicitly?
- Are you rebuilding the input each call instead of relying on a chat session?
Walking through these in order resolves most "my reference image got ignored" reports within an hour. Hope this saves you the afternoon I lost to it.