Cutting Gemini Embedding's output_dimensionality from 768 to 256 reduced my vector DB storage to one-third

Trimming 768 to 256 in code

The code change is small. Add output_dimensionality to EmbedContentConfig and you are done.

# Convert wallpaper metadata into a 256-dimensional embedding
from google import genai
from google.genai import types
import numpy as np
 
client = genai.Client(api_key="YOUR_GEMINI_API_KEY")
 
def embed_text(text: str, dim: int = 256) -> list[float]:
    """Return a semantic embedding at the requested dimensionality."""
    result = client.models.embed_content(
        model="gemini-embedding-001",
        contents=text,
        config=types.EmbedContentConfig(
            output_dimensionality=dim,
            task_type="SEMANTIC_SIMILARITY",
        ),
    )
    return result.embeddings[0].values
 
# L2 normalize before storing — almost always required for cosine similarity
def l2_normalize(v: list[float]) -> list[float]:
    arr = np.array(v, dtype=np.float32)
    norm = np.linalg.norm(arr)
    if norm == 0:
        return v
    return (arr / norm).tolist()
 
vec = l2_normalize(embed_text("Illustration of blue cherry blossoms under a night sky"))
print(len(vec))   # Expected: 256
print(vec[:3])    # Expected: first three of 256 floats

You could also fetch 768 dimensions and slice with vec[:256]. The math comes out the same, but you waste response bandwidth and JSON parsing time. I prefer to ask the API for 256 from the start.

One thing to be aware of: the input token count drives Gemini Embedding's price, not the output dimensionality. Asking for 256 instead of 768 does not save you any API spend. The savings show up downstream — in storage, in network egress when you ship the vector to a search service, and in the size of any HNSW or IVF index you build on top. For an indie project those downstream costs were where my actual bill was hiding.

Recall held up better than I expected

I evaluated recall@10 across 200 real search queries on the same 80,000 wallpaper rows. I treated the 768-dimensional results as the ground truth and measured how often the top 10 from each smaller embedding overlapped.

• 768 dimensions: recall@10 = 0.985 (baseline)
• 512 dimensions: recall@10 = 0.978 (-0.7 points)
• 256 dimensions: recall@10 = 0.972 (-1.3 points)
• 128 dimensions: recall@10 = 0.948 (-3.7 points)

The numerical gap between 256 and 128 looks small on paper. In the actual app, however, scrolling through "similar wallpapers" at 128 dimensions started showing genres bleeding into each other, especially for short-titled items. My guess is that shorter text leaves the model less room to pack meaning into the leading dimensions, and at 128 that headroom finally runs out.

At 256 I could not tell the difference from 768 by eye. At 128 I could. The line I follow on my own products is simple: if I, as the developer who looks at this every day, can spot the regression, I do not ship it.

A second observation worth recording: the recall numbers above were measured against the 768-dimensional results. That is convenient, but it implicitly assumes 768 is the ground truth. When I separately compared all four configurations against a small hand-labeled set of 80 query/result pairs, the pattern shifted slightly. 256 dimensions was within 1.5 points of 768, and 128 dimensions trailed by about 4 points. The story did not really change, but it reminded me to label at least a few queries by hand whenever I retire an embedding model — automated metrics drift, and a small human-labeled set is the quickest way to catch it.

A decision matrix for picking the dimensionality

People keep asking me which dimensionality they should pick, so here is the rule of thumb I now follow on my own indie projects:

• Under 10,000 vectors, lightweight search only → start at 256, climb to 512 only if you measure a real regression
• 10,000 to 100,000 vectors, recommendations or related-content → 256 dimensions (recommended) plus float16 storage
• 100,000 to 1 million vectors, accuracy matters → 256 for first-stage retrieval, full 768 only for a tight re-ranking step
• Over 1 million vectors, cost is the priority → 256 dimensions plus int8 quantization
• Legal, medical, or any domain where a single correct answer exists → keep 768 and absorb the cost with an efficient backend such as pgvector

If most of your inputs are short — titles, tags, search keywords — I would lean one notch higher than the rule above. My wallpaper app is mostly short text, so I am deliberately conservative.

Storage tricks that actually paid off

Just dropping the dimensionality to 256 brings the raw vectors down to 80,000 × 256 × 4 = ~80 MB. Including indexes and metadata, the total settled around 700 MB — roughly one-third of the original. Two further tricks earned their keep.

The first is float32 → float16. It is nearly lossless and halves the vector size.

# float16 quantization — nearly lossless, half the storage
import numpy as np
 
def to_float16_blob(vec_f32: np.ndarray) -> bytes:
    """Pack a float32 vector into a float16 byte string for storage."""
    return vec_f32.astype(np.float16).tobytes()
 
def from_float16_blob(blob: bytes, dim: int = 256) -> np.ndarray:
    """At query time, restore the float32 vector from the float16 byte string."""
    return np.frombuffer(blob, dtype=np.float16, count=dim).astype(np.float32)
 
# Usage
v = np.random.randn(256).astype(np.float32)
v /= np.linalg.norm(v)            # Assumes L2-normalized input
blob = to_float16_blob(v)         # Expected size: 512 bytes (256 * 2)
restored = from_float16_blob(blob)
print(np.dot(v, restored))        # Expected: very close to 1.0

The second trick is int8 quantization, which cuts storage by another factor of four. The catch is that you must persist a per-row scale factor; without it the dot products land far from cosine similarity and the ranking falls apart.

# int8 quantization — keep the per-row scale or rankings break
import numpy as np
 
def quantize_int8(vec_f32: np.ndarray) -> tuple[bytes, float]:
    """Scale an L2-normalized float32 vector into the -127..127 int8 range."""
    scale = float(np.max(np.abs(vec_f32)))
    if scale == 0.0:
        return b"\x00" * len(vec_f32), 1.0
    quantized = np.clip(np.round(vec_f32 / scale * 127), -127, 127).astype(np.int8)
    return quantized.tobytes(), scale
 
def dequantize_int8(blob: bytes, scale: float, dim: int = 256) -> np.ndarray:
    """Restore the float32 vector from int8 bytes and a stored scale factor."""
    arr = np.frombuffer(blob, dtype=np.int8, count=dim).astype(np.float32)
    return arr * (scale / 127.0)
 
# Usage
v = np.random.randn(256).astype(np.float32)
v /= np.linalg.norm(v)
blob, scale = quantize_int8(v)    # blob is 256 bytes, scale is a float stored alongside
restored = dequantize_int8(blob, scale)
print(np.dot(v, restored))        # Expected: above 0.99 (close to lossless)

With this in place, recall@10 dropped a further 0.005 points and the difference vanished in visual review. I store the scale in a vec_scale REAL column next to the vector. It looks like a tiny detail — one float per row — but forgetting it is the bug that costs you a weekend.

Five steps for actually rolling this out

Trying to "just rewrite all 80,000 vectors at 256 dimensions overnight" is the easiest way to lose a long weekend. After getting bitten once, I follow this sequence:

1. Run both columns side by side. Add embedding_256 next to the existing embedding_768 on the same row. Put the query path behind a flag so you can flip back instantly.
2. Index new inserts at 256 first. Touch nothing old. Let new rows flow into embedding_256 for a day or two so the write path stabilizes before any backfill begins.
3. A/B compare reads in the dark. For each user query, run both 768 and 256 internally and log the top-10 diff. Serve 768 to production while you eyeball the 256 results daily.
4. Backfill carefully. Re-embed historical rows in batches with exponential backoff. The one time I skipped backoff on a nightly job, a 429 storm cost me tens of thousands of rows that had to be reprocessed.
5. Flip reads, watch metrics, drop the old column last. Switch the query path to 256, wait several days, confirm CTR and bounce rates are stable, then DROP the 768 column. Doing the DROP last is the part that actually matters.

This sequence kept the rollout boringly uneventful, which is exactly the property you want from a storage migration.

A 60-second CI benchmark gate

Every time the embedding dimensionality or the quantization scheme changes, I run this tiny benchmark in CI. It keeps a fixed set of queries and the expected top-3 IDs, and fails the build if the ranking drifts past a floor.

# tests/embedding_gate.py — runs in under a minute in CI
import numpy as np
 
GOLDEN = [
    ("blue cherry blossoms at night", ["sakura_night_blue_001", "sakura_blue_pixel_002", "starry_night_sakura_003"]),
    ("pale watercolor sunflower", ["sunflower_watercolor_010", "sunflower_pale_011", "summer_watercolor_012"]),
    ("simple japanese wave pattern", ["wave_japanese_seigaiha", "wave_japanese_blue", "minimal_wave_indigo"]),
]
RECALL_FLOOR = 0.85  # Fraction of expected IDs that must appear in the top-N
TOP_N = 3
 
def assert_embedding_quality(load_index, search) -> None:
    """Given an index loader and a search callable, fail loudly if recall regresses."""
    index = load_index()
    hits = 0
    total = 0
    for query, expected_ids in GOLDEN:
        result_ids = [doc.id for doc in search(index, query, top_n=TOP_N)]
        for eid in expected_ids:
            if eid in result_ids:
                hits += 1
            total += 1
    recall = hits / total
    assert recall >= RECALL_FLOOR, f"embedding gate failed: recall={recall:.3f}"
 
# Usage (call from pytest or CI with the new index swapped in)
# assert_embedding_quality(load_index=lambda: build_index_v256_int8(), search=ann_search)

Three queries is small, but it has already caught a "forgot to persist the int8 scale factor" bug for me once. The trick is to put only results you absolutely cannot ship without into GOLDEN. Keep the set small enough that you actually maintain it.

Gotchas an indie developer is likely to hit

A handful of pitfalls I either tripped over or almost tripped over:

• If you slice with vec[:256], both sides — the writer and the searcher — must agree on the length. A short writer and a long reader will not throw; they will silently produce broken dot products.
• L2 normalization can happen before or after quantization, but pick one and stick with it. Mixing the two on either side of production caused the worst ranking bug I had to debug.
• Forgetting the per-row int8 scale factor is the canonical way to break int8 quantization. The directions are preserved; the magnitudes are not.
• Many ANN libraries assume float32 under the hood. Handing them int8 bytes without dequantizing can compile cleanly and rank wrongly at the same time. CI is the only reliable defense.
• Run exponential backoff on the backfill or you will get a cascade of 429s on a nightly batch and lose rows.
• When the backend hits a row-count ceiling before it hits a storage ceiling — Cloudflare D1 is the canonical example — split out the vector store earlier rather than later. I ultimately moved my bigger apps to pgvector.

For the actual backends, the sqlite-vec setup for very lightweight RAG is my default at indie scale; when the index grows past a few hundred thousand rows, pgvector on Postgres is what I move to. If you want everything to run in the browser, putting Gemini embeddings into IndexedDB for a serverless FAQ search is another path worth knowing.

What I would try first

If you are running 768-dimensional embeddings in production, take 1,000 random rows, regenerate them at output_dimensionality=256, and visually compare the cosine top-10 against your current results. The whole experiment fits inside an hour, and on the other side of it you may find that your monthly storage bill was always asking to be cut to a third. Once the 256 path is settled, layer in float16 first and int8 second, in that order, so you always have a step you can roll back to.

Thank you for reading this far. If a similar slow-burn storage cost has been quietly chipping away at your indie project, I hope this small write-up gives you something concrete to try.