Three Places Gemini API Embedding Broke on Me — and What Actually Fixed Them

Somewhere past the 300th item, INVALID_ARGUMENT started coming back. The job was simple enough: embed wallpaper descriptions, then auto-assign each one to a category. It had run clean across a few dozen items, so my first instinct was that a malformed row had slipped into the data.

The real cause sat further upstream. Not the call itself, but a mismatch between how I built the index and what I changed afterward. Working on my own app's backend, I keep running into decisions I made weeks earlier and promptly forgot about.

Three things had to be sorted out before the pipeline ran reliably: how dimensions are handled, rate limits, and retrieval quality. None of them announced themselves clearly in the error output, so here they are in the order I worked through them.

INVALID_ARGUMENT — Dimensions Aren't a Dial You Turn Later

Embedding dimensionality is fixed on some models and configurable on others. text-embedding-004 returns 768 dimensions, full stop. Models like gemini-embedding-001 let you pick from 3072 / 1536 / 768 via output_dimensionality. Ignoring that difference is what broke my pipeline.

What I actually did: built the index at 3072 dimensions, then tried to cut costs by swapping in text-embedding-004. Two separate failures fired at once. Calls that requested a dimension the model doesn't support were rejected by the API, and vectors whose length didn't match the collection were rejected by the vector store. Both come back worded like an argument problem, so I spent half an hour treating them as one bug.

The fastest way through is to pin down where the error originated before touching any code.

The fix is to treat the model and the dimension as properties of the index, not of the call. I moved both into constants at the top of the module so nothing downstream can pick a different value.

from google import genai
from google.genai import types
 
client = genai.Client(api_key="YOUR_GEMINI_API_KEY")
 
# Locked to the index. Changing either means a full regeneration.
EMBED_MODEL = "gemini-embedding-001"
EMBED_DIM = 768
 
# ❌ Passing an unsupported dimension to a fixed-size model → INVALID_ARGUMENT
resp = client.models.embed_content(
    model="text-embedding-004",          # 768 only
    contents="Beautiful mountain landscape",
    config=types.EmbedContentConfig(output_dimensionality=3072),
)
 
# ✅ Use a model that supports dimension selection, with the index's value
resp = client.models.embed_content(
    model=EMBED_MODEL,
    contents="Beautiful mountain landscape",
    config=types.EmbedContentConfig(
        output_dimensionality=EMBED_DIM,
        task_type="RETRIEVAL_DOCUMENT",
    ),
)
vector = resp.embeddings[0].values

On choosing a dimension: at the data volumes an indie developer actually works with, the retrieval quality gap between 768 and 3072 can be small enough that you won't feel it. I rebuilt the wallpaper index at 768 over a weekend batch. Storage and transfer costs dropped in a straightforward way, and starting small leaves you more room to change your mind later.

If You Truncate the Vector, Re-Normalize It

Models that let you request a smaller dimension work by using the leading slice of a longer vector. What's easy to miss is that a truncated vector no longer has an L2 norm of 1.

Whether that matters depends on your distance function. With cosine distance, normalization happens inside the comparison, so rankings don't shift. With inner product or L2 distance, differences in norm feed straight into the scores. I dropped the dimension without checking my ChromaDB distance setting and spent a few days in that vague "retrieval feels worse now" state.

One function in the regeneration batch is enough.

import numpy as np
 
def l2_normalize(vec: list[float]) -> list[float]:
    """Bring a truncated embedding back to unit length."""
    arr = np.asarray(vec, dtype=np.float32)
    norm = float(np.linalg.norm(arr))
    if norm == 0.0:
        return arr.tolist()
    return (arr / norm).tolist()
 
 
# Quick sanity check: full-size vectors sit near 1.0, truncated ones drift
raw = resp.embeddings[0].values
print(round(float(np.linalg.norm(raw)), 4))
print(round(float(np.linalg.norm(l2_normalize(raw))), 4))  # 1.0

Be consistent across both sides. Normalizing documents but not queries — or the reverse — produces the least obvious kind of ranking damage when your store uses inner product.

RESOURCE_EXHAUSTED (429) — I Lost 400 Items and Started Over

Pushing 500 descriptions through in one pass produced a steady stream of RESOURCE_EXHAUSTED. I assumed a 50 ms interval would be plenty; on the free tier you hit the per-minute ceiling almost immediately.

Exponential backoff got the job through, but the implementation had a different weakness. At item 400 an exception escaped, and every vector held in memory went with it. Re-running those 400 items is what convinced me to design for partial progress instead of hoping the run survives.

What I use now combines full-jitter backoff, a concurrency ceiling, and a checkpoint file appended one line at a time.

import asyncio
import json
import random
from pathlib import Path
 
from google import genai
from google.genai import types
 
client = genai.Client(api_key="YOUR_GEMINI_API_KEY")
 
CHECKPOINT = Path("embed_checkpoint.jsonl")
MAX_CONCURRENCY = 4
 
 
def load_done_ids() -> set[str]:
    if not CHECKPOINT.exists():
        return set()
    return {
        json.loads(line)["id"]
        for line in CHECKPOINT.read_text(encoding="utf-8").splitlines()
        if line.strip()
    }
 
 
async def embed_one(item: dict, sem: asyncio.Semaphore, max_retries: int = 5) -> dict | None:
    async with sem:
        for attempt in range(max_retries):
            try:
                resp = await client.aio.models.embed_content(
                    model=EMBED_MODEL,
                    contents=item["text"],
                    config=types.EmbedContentConfig(
                        output_dimensionality=EMBED_DIM,
                        task_type="RETRIEVAL_DOCUMENT",
                    ),
                )
                vec = l2_normalize(resp.embeddings[0].values)
                return {"id": item["id"], "vector": vec}
            except Exception as e:
                msg = str(e)
                if "429" in msg or "RESOURCE_EXHAUSTED" in msg:
                    # Full jitter: uniform 0..2^attempt seconds, so retries don't sync up
                    await asyncio.sleep(random.uniform(0, 2 ** attempt))
                    continue
                raise
        return None
 
 
async def embed_all(items: list[dict]) -> int:
    done = load_done_ids()
    todo = [i for i in items if i["id"] not in done]
    sem = asyncio.Semaphore(MAX_CONCURRENCY)
    written = 0
 
    with CHECKPOINT.open("a", encoding="utf-8") as f:
        tasks = [embed_one(i, sem) for i in todo]
        for coro in asyncio.as_completed(tasks):
            rec = await coro
            if rec is None:
                continue
            f.write(json.dumps(rec, ensure_ascii=False) + "\n")
            f.flush()  # keep everything up to the moment it dies
            written += 1
 
    return written

Calling f.flush() on every line looks wasteful, but whatever sits in the buffer when a batch dies is gone. At a few hundred items the write cost is irrelevant, so I take the safe side.

Estimating runtime got easier too. Serially, with a one-second gap, 512 items spend 512 seconds sleeping alone — close to ten minutes once API latency is added. At a concurrency of 4 the theoretical floor is roughly a quarter of that, and with backoff waits mixed in you land somewhere in the three-to-four minute range. Raising the ceiling too far just increases 429s and slows things down, so I start at 4 and go to 6 only if I need to.

Any item that failed simply isn't in the checkpoint, so the next run picks it up. Making re-runs idempotent lowered the mental cost of scheduling a batch overnight more than I expected.

Missing task_type — Results That Don't Line Up

Once the errors stopped, one problem remained: searching for "natural mountain scenery" still surfaced unrelated wallpapers. The cause was a missing task_type.

Gemini's embedding models use different vector spaces depending on the intended task. If queries and stored documents don't declare their respective types, both land in a generic space and retrieval suffers.

# ❌ No task_type — generic embedding space
resp = client.models.embed_content(
    model=EMBED_MODEL,
    contents="Looking for a natural wallpaper",
    config=types.EmbedContentConfig(output_dimensionality=EMBED_DIM),
)
 
# ✅ Incoming search queries: RETRIEVAL_QUERY
query = client.models.embed_content(
    model=EMBED_MODEL,
    contents="Looking for a natural wallpaper",
    config=types.EmbedContentConfig(
        output_dimensionality=EMBED_DIM,
        task_type="RETRIEVAL_QUERY",
    ),
)
 
# ✅ Documents you store: RETRIEVAL_DOCUMENT
doc = client.models.embed_content(
    model=EMBED_MODEL,
    contents="Sunlight filtering through green forest leaves",
    config=types.EmbedContentConfig(
        output_dimensionality=EMBED_DIM,
        task_type="RETRIEVAL_DOCUMENT",
    ),
)

The useful property of this fix is that you can change the query side alone and watch what happens. Regenerating the document side means rebuilding the whole index, so it makes sense to look for a signal on the cheap side first.

Measuring the Improvement Instead of Sensing It

When I first added task_type, my evaluation method was typing a few phrases into the search box and deciding it felt better. Days later I made another change and could no longer say which one had helped. That's when I built a 20-question ground-truth set.

The mechanics are unremarkable: pair each query with the wallpaper IDs that should rank highly, then measure recall@5.

import numpy as np
 
# Hand-built ground truth. Twenty cases is enough to see the direction of a change.
GOLD = [
    {"query": "natural scenery", "relevant": ["w-102", "w-118", "w-233"]},
    {"query": "city lights at night", "relevant": ["w-045", "w-291"]},
    # ... up to roughly twenty
]
 
 
def search(query_vec: list[float], index: dict[str, list[float]], k: int = 5) -> list[str]:
    q = np.asarray(query_vec, dtype=np.float32)
    scored = [
        (doc_id, float(np.dot(q, np.asarray(vec, dtype=np.float32))))
        for doc_id, vec in index.items()
    ]
    scored.sort(key=lambda x: x[1], reverse=True)
    return [doc_id for doc_id, _ in scored[:k]]
 
 
def recall_at_k(index: dict[str, list[float]], embed_query, k: int = 5) -> float:
    scores = []
    for case in GOLD:
        ranked = search(embed_query(case["query"]), index, k)
        hit = len(set(ranked) & set(case["relevant"]))
        scores.append(hit / len(case["relevant"]))
    return sum(scores) / len(scores)

Because the stored vectors are normalized, the dot product is already cosine similarity. Running task_type on and off through this harness moved recall@5 from the low 0.6s into the high 0.7s on my twenty cases. The sample is small enough that the absolute numbers are indicative at best — but being able to confirm that reverting the change lowers the score is what makes decisions fast.

Saying retrieval improved by 15–20% only became defensible once those twenty cases existed. Before that, my evidence was my own memory of last week, which is a poor basis for choosing what to do next. The harness took about an hour to build and has been paying for itself in decision time ever since.

Model Choice Is an Index Decision, Not a Call-Site Decision

Experimental embedding models tend to look attractive on benchmark tables. In production I stay on the stable one — not because of an accuracy gap, but because switching costs more than editing one line.

Change the model and the vector space changes with it, which means regenerating every document and rebuilding the index. Structurally, it's the same decision as changing dimensions. At 500 items that's a few minutes overnight. At tens of thousands, you also have to decide how retrieval quality holds up while the regeneration runs.

This family of models moves quickly, and newer embedding lines are already showing up behind features like File Search. Model IDs, supported dimensions, and pricing all shift, so plan on checking primary sources before you commit. My compromise is a single constants block holding the model and dimension: touching that block is the signal that a regeneration plan is needed.

# Config pinned in the wallpaper app backend (for reference)
EMBEDDING_CONFIG = {
    "model": EMBED_MODEL,                # change → regenerate the index
    "output_dimensionality": EMBED_DIM,  # same
    "normalize": True,                   # store truncated vectors at unit length
    "query_task_type": "RETRIEVAL_QUERY",
    "doc_task_type": "RETRIEVAL_DOCUMENT",
    "max_concurrency": 4,                # lower it if 429s climb
    "max_retries": 5,                    # with full jitter
}

Two Things That Moved the Needle Further

After task_type, two experiments survived contact with the harness.

The first was embedding descriptions rather than bare category names. "Wallpapers featuring mountains, forests, lakes, and open skies" holds up against real search phrasing far better than the single word "nature."

The second was preprocessing the query. What users type is short and highly variable, so expanding it into a brief descriptive sentence with a quick Gemini call before embedding produced steadier rankings. That adds a round trip, so it's a trade between perceived speed and precision. Wallpaper search tolerates a few hundred milliseconds, so I took the precision.

Start With task_type

If you're stuck on any of this right now, adding RETRIEVAL_QUERY on the query side is the cheapest move available. No index rebuild, and it's reversible in minutes. Once you see an effect, build the twenty-case set before making the next change — that ordering is what stopped me from backtracking.

Pin the model and dimension up front, re-normalize anything you truncate, and give your batches a checkpoint. With those three in place, Gemini Embedding stopped being the thing that kept me up late.