Define What Counts as a Duplicate Before Adding Embedding Search to Your Image Pipeline

Measuring Where Perceptual Hashing Actually Ends

How much of layer 2 can a hash really cover? That is testable in an afternoon. I generated a structured test image, applied the transformations that actually occur in my pipeline, and measured dHash Hamming distances. This is the exact code I used.

from PIL import Image
 
def dhash(img, size=8):
    # Shrink to 9x8 grayscale, encode neighbor brightness gradients into 64 bits
    g = img.convert("L").resize((size + 1, size), Image.LANCZOS)
    px = g.load()
    bits = 0
    for y in range(size):
        for x in range(size):
            bits = (bits << 1) | (1 if px[x, y] > px[x + 1, y] else 0)
    return bits
 
def hamming(a, b):
    # 0 = identical. Closer to 64 = unrelated
    return bin(a ^ b).count("1")
 
# Usage
a = dhash(Image.open("original.png"))
b = dhash(Image.open("candidate.jpg"))
print(hamming(a, b))  # Expected: single digits for transforms, 20+ for unrelated images

The measurements:

Everything typical of layer 2 — re-encodes, resizes, light crops — lands at single-digit distances. Because the hash normalizes size before comparing, resizes and even aspect-ratio stretches come back at distance 0. As a layer 2 gatekeeper, dHash earns its position.

No Threshold Separates 29 From 28

Two results contradicted my expectations, and they carry the article.

First: cropping away the bottom 25% produced a distance of 29 — statistically indistinguishable from the completely unrelated image at 28. Heavy crops look like different images to a hash. You might think raising the threshold to 30 would catch them, but at that point unrelated images start getting flagged too. There is a boundary between 29 and 28 that no threshold tuning can cross. If you need to catch heavily cropped copies, you need a semantic eye — which is exactly where embeddings belong.

Second: swapping the color channels wholesale moved the distance by only 3. dHash reads grayscale gradients, so identical composition means identical hash, regardless of palette. My two alternate printings were flagged as one image by precisely this property.

And here is the part worth sitting with. Had I switched the gate to embedding search, those two prints would have scored highly similar as well — they are close both visually and semantically. For layer 3, hashes and embeddings return the same answer. Neither technology would have resolved the disagreement I started with. What looked like a technology-selection problem was a definition problem wearing a disguise. That realization was the single most useful output of this whole exercise.

What Changed on the File Search Side

What convinced me to adopt embeddings in the suggest-only role was this year's sequence of updates. According to the Gemini API release notes, the multimodal embedding model gemini-embedding-2 entered preview on March 10, 2026, went GA on April 22, and on May 5 File Search gained native image embedding through that model. No more OCR detour — images go into the index as images.

You could build your own vector store and nearest-neighbor search, but for a solo-maintained collection in the tens of thousands, letting File Search own the index is easier to keep alive. I covered the search-side experience in Letting File Search's Multimodal Mode Find Wallpapers I Couldn't; this project simply repurposes that same index to surface duplicate suspects.

One operational note. Every embedded image costs an API call, and embedding models retire on Google's schedule, not yours. A local dHash has no sunset date: at a scale of 100,000 assets the comparison still runs entirely on my machine at an API cost of exactly zero. For a gate that every single asset must pass through, I prefer the component that cannot be shut down, repriced, or re-versioned underneath me. When the embedding side does turn over, the playbook from The Day You Switch Gemini Embedding Models: Designing a Zero-Downtime Reindex applies as-is.

The Final Shape: Machines Hold, Humans Decide

The registration flow settled into the code below. The critical detail is that a dHash hit never deletes anything — it queues the candidate with its evidence attached.

import hashlib
import json
import time
from pathlib import Path
from PIL import Image
 
DHASH_THRESHOLD = 10  # My layer 2 transforms all measured at distance 8 or less
 
def sha256_of(path):
    return hashlib.sha256(Path(path).read_bytes()).hexdigest()
 
def check_new_image(path, index):
    """index: dict of existing assets, {id: {"sha": ..., "dhash": ...}}"""
    sha = sha256_of(path)
    h = dhash(Image.open(path))
 
    for asset_id, entry in index.items():
        if entry["sha"] == sha:
            return {"action": "reject", "reason": f"identical file: {asset_id}"}
 
    suspects = []
    for asset_id, entry in index.items():
        d = hamming(h, entry["dhash"])
        if d <= DHASH_THRESHOLD:
            suspects.append({"asset_id": asset_id, "distance": d})
 
    if suspects:
        # Never delete here. Queue it with evidence for a human
        queue_item = {
            "path": str(path),
            "suspects": sorted(suspects, key=lambda s: s["distance"]),
            "queued_at": int(time.time()),
        }
        Path("review_queue.jsonl").open("a").write(
            json.dumps(queue_item, ensure_ascii=False) + "\n"
        )
        return {"action": "hold", "suspects": len(suspects)}
 
    return {"action": "accept", "sha": sha, "dhash": h}

The review screen for that queue also shows the top five nearest neighbors from File Search. Once you state plainly that embeddings return "similar things" rather than "duplicates," their false positives stop being a problem — series pieces and same-subject works clustering together actually makes the layer 3 editorial call easier, not harder.

It echoes a lesson from when I expanded my category taxonomy and had to bound the blast radius of reclassification (The Day We Went From 30 Categories to 34): the scope of irreversible actions you delegate to a machine should be derived from the damage of its worst mistake, not from its average accuracy.

Generalizing the Decision

If you are weighing a dedup check for your own collection, or wondering whether embedding search should join it, the decision compresses into this table.

The order matters:

1. Write down your layer 3 edge cases and settle the definition first.
2. Measure layer 2 thresholds by applying real transformations to your own material.
3. Only then add embedding search, strictly as a suggestion layer.

Done in that order, the check is useful at every intermediate stage. Done in reverse — embeddings first, definition never — you end up staring at a ranked list of high similarity scores while nobody can answer the only question that matters: which of these am I allowed to delete?

Start by Writing the Definition Down

The next step I would suggest costs nothing: before writing any code, list three real pairs from your own collection where you have genuinely hesitated over whether something was a duplicate. Decide which of the three layers each pair belongs to. The thresholds, the placement of embedding search, and the authority you grant the machine will all follow from those three decisions. I put the definition off until two alternate printings stopped me at the gate; if this writeup spares you that detour, it has done its job.