⬡ Advanced/2026-07-31Advanced

After the Vertex AI Rename, I Measured My Search Stack. Ranking Held. Aggregation Did Not.

When Vertex AI became Gemini Enterprise Agent Platform, I benchmarked my own docs search. Ranking barely moved. The exact-match layer quietly lost most of its results without raising a single error.

Gemini Enterprise Agent Platform Vertex AI¹² search design aliasing operations¹²

✦ Premium Article

I found the line in the release notes late on July 30th: Vertex AI is now Gemini Enterprise Agent Platform.

My first thought went to my runbooks. Running six apps solo means my personal notes and my in-app help index have slowly turned into a small knowledge base. Every "Vertex AI" in there now points at a name that no longer exists.

Tomorrow, when someone searches with the new name, would anything come back at all?

So the next morning I measured it. The answer was the opposite of what I expected. Ranking held up fine. Something else had gone quietly missing.

What I Measured

I rebuilt my situation as the smallest thing that still reproduces it — no external services, so anyone can get the same numbers.

Corpus: 30 runbook fragments. Twelve of them describe platform-side concerns (deployment, quota, IAM, billing). Ten of those use the legacy name, two use the new one.
Retriever: BM25 (k1=1.5, b=0.75). Japanese text tokenized as 2-grams, ASCII as whole words.
Queries: three families of five.
- Topical — "how do I handle a quota overage" — the query carries signal beyond the product name.
- Name-only — "what is X", "X settings" — the name is the only handle.
- Unrelated — cache TTL, AdMob — a control group.

Three responses, compared head to head:

Condition	Action	Reindex needed?
A: do nothing	Leave the legacy name in place	No
B: bulk rewrite	Replace the old name inside every chunk, reindex	Yes
C: query-side expansion	Leave the index alone, expand queries in both directions	No

B looks obviously right. It brings your data in line with reality. That was my assumption going in.

The Benchmark Code

No dependencies. Swap in your own corpus and you can run exactly this.

# -*- coding: utf-8 -*-
"""Measuring how a product rename affects lexical retrieval. No external deps."""
import math, re
from collections import Counter
 
OLD = "vertex ai"
NEW = "gemini enterprise agent platform"
 
def tok(s: str):
    """2-grams for CJK, whole words for ASCII. Keeps the BM25 vocabulary consistent."""
    s = s.lower()
    s = re.sub(r"[、。（）()「」・,.]", " ", s)
    parts = []
    for w in s.split():
        if re.fullmatch(r"[a-z0-9_\-\.]+", w):
            parts.append(w)               # ASCII stays a single token
        else:
            for i in range(len(w) - 1):
                parts.append(w[i:i + 2])  # CJK becomes 2-grams
            if len(w) == 1:
                parts.append(w)           # don't drop single-character tokens
    return parts
 
class BM25:
    def __init__(self, docs, k1=1.5, b=0.75):
        self.ids = [d[0] for d in docs]
        self.toks = [tok(d[1]) for d in docs]
        self.k1, self.b = k1, b
        self.N = len(docs)
        self.avgdl = sum(len(t) for t in self.toks) / self.N
        self.df = Counter()
        for t in self.toks:
            self.df.update(set(t))        # document frequency counts sets, not totals
        self.tf = [Counter(t) for t in self.toks]
 
    def search(self, q: str, topk=5):
        qt = tok(q)
        scores = []
        for i in range(self.N):
            s, dl = 0.0, len(self.toks[i])
            for w in qt:
                f = self.tf[i].get(w, 0)
                if not f:
                    continue
                idf = math.log(1 + (self.N - self.df[w] + 0.5) / (self.df[w] + 0.5))
                s += idf * (f * (self.k1 + 1)) / (
                    f + self.k1 * (1 - self.b + self.b * dl / self.avgdl))
            scores.append((s, self.ids[i]))
        # Without a deterministic tiebreak your measurements drift between runs
        scores.sort(key=lambda x: (-x[0], x[1]))
        return [i for s, i in scores[:topk] if s > 0]
 
def expand_bidirectional(q: str) -> str:
    """Condition C. Old to new AND new to old. One direction only saves half your users."""
    ql = q.lower()
    if OLD in ql:
        return q + " " + NEW
    if NEW in ql:
        return q + " " + OLD
    return q
 
def precision_at_k(idx: BM25, queries, gold: set, k=5, expand=None):
    total = 0.0
    for q in queries:
        got = idx.search(expand(q) if expand else q, k)
        if not got:
            continue
        total += sum(1 for d in got if d in gold) / k
    return round(total / len(queries), 3)

I wrote expand_bidirectional one-way first — legacy queries rewritten toward the new name. That version dropped every reader who had already adopted the new terminology. Expansion has to run both ways or it is not worth shipping.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦A reproducible BM25 benchmark of retrieval quality immediately after a vendor product rename

✦Why bulk-rewriting your chunks is a net loss (P@5 drops from 0.92 to 0.08 for legacy-name queries)

✦The layer that actually breaks is exact-match routing and aggregation, plus where to put the alias resolver instead

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

The First Surprise

I started with Recall@3: does a correct document appear anywhere in the top three?

For topical queries, all three conditions scored 1.000. No difference whatsoever.

In hindsight the reason is plain. "How do I handle a quota overage" carries "quota", "overage", "handle". Those terms have lower document frequency than "vertex" or "ai", so BM25 weights them higher. The product name was never doing the discriminating work.

I had assumed the scary part of a rename was retrieval failure. For any query that carries a topic, the name is a supporting actor.

Tightening the metric to P@5 — how many of the top five are actually relevant — finally separates the conditions.

Query type	A: do nothing	B: bulk rewrite	C: query expansion
Name-only, legacy name	0.920	0.080	0.920
Name-only, new name	0.440	1.000	0.920
Topical, legacy name	0.760	0.400	0.840
Topical, new name	0.680	0.840	0.840

Doing nothing has exactly one weak spot: 0.440 on name-only queries using the new terminology. Barely two of five results are relevant.

The Second Surprise

Look at column B.

The bulk rewrite pushes new-name queries to a perfect 1.000. Then the same edit drops legacy-name queries from 0.920 to 0.080. Topical legacy queries fall from 0.760 to 0.400 — roughly half.

Nobody stops using the old name on rename day. Existing users are precisely the population most attached to the old vocabulary. In my own help search, I would expect months before the new phrasing dominates.

So B is a trade where you sacrifice the readers you have for readers who have not arrived yet. That is a net loss, not a migration.

Condition C lands everything in a 0.840–0.920 band. It never wins a column outright, and it never collapses in one either. It also requires no reindex, which means it can ship the same day the rename lands.

C is not free. Averaged over 200 runs, a plain query took 102.7 µs and an expanded one 125.6 µs — about 22% more. The relative number is not nothing, but 23 µs of absolute overhead disappears into any architecture with a network hop in it. Measure it against your own stack before deciding it matters.

What Actually Broke

Everything above concerns retrieval. Here is the number that made me stop.

Take the twelve platform-side documents and apply a naive exact-match gate:

Predicate	Matches out of 12
`"Vertex AI" in text` (pinned to legacy name)	10
`"Gemini Enterprise Agent Platform" in text` (pinned to new name)	2
Alias set	12

The moment you update that string to the new name, ten of twelve documents vanish. And unlike search, this layer hands back two plausible-looking results, so nothing announces that it emptied out.

A ranker degrades visibly. Something wrong stays near the top and a human notices. An exact-match gate returns two rows and exits zero.

I counted the places in my own setup with that shape:

Cost aggregation — monthly billing grouped by product name. Renamed spend silently falls into "other".
Alert routing — notification targets chosen by matching product names in log messages. Unmatched alerts flow to the default channel and get buried.
Document tagging — a batch job that files runbooks by product. Only documents written after the rename land in a different bucket.
A CI banned-terms check — the very mechanism meant to catch deprecated product names had those names hardcoded.

None of the four raise an exception. The numbers just get a little smaller. This is the kind of breakage you discover weeks later, staring at a monthly report that feels slightly off.

Where the Alias Layer Belongs

The measurements make the placement decision concrete.

Layer	Rename impact	Will you notice?	Fix
Ranking (BM25, vector search)	Low to moderate	Yes, by eye	Bidirectional query expansion
Exact-match gates and filters	High	No	Normalize before comparing
Aggregation and grouping	High	Weeks later	Group on a canonical key
Chunk body text	—	—	Leave it alone

That last row is the conclusion I came away with. Document bodies should keep the name they were written with. A chunk is a record of what was true when someone wrote it, not a surface to repaint every time marketing moves.

Name resolution belongs to the reader, not the record. Structure it that way and the next rename costs you nothing at index time.

Implementation

One source of truth for aliases, consumed by all three layers.

# -*- coding: utf-8 -*-
"""Single source of truth for product aliases: search, filtering and aggregation all read this."""
from dataclasses import dataclass
from datetime import date
 
@dataclass(frozen=True)
class ProductAlias:
    canonical: str                    # the aggregation key, distinct from any display name
    surfaces: tuple                   # every spelling that actually appears in docs or queries
    renamed_on: date | None = None    # rename date, kept so aliases can expire
    note: str = ""
 
ALIASES: list[ProductAlias] = [
    ProductAlias(
        canonical="google.ai.platform",
        surfaces=("vertex ai", "vertex-ai", "vertexai", "aiplatform",
                  "gemini enterprise agent platform"),
        renamed_on=date(2026, 7, 30),
        note="Renamed from Vertex AI on 2026-07-30. Keep matching the old name for now.",
    ),
    ProductAlias(
        canonical="google.ai.studio",
        surfaces=("ai studio", "google ai studio", "aistudio"),
    ),
]
 
# Build the reverse map once at startup
_SURFACE_TO_CANONICAL = {
    s: a.canonical for a in ALIASES for s in a.surfaces
}
# Longest surface first, otherwise short aliases eat substrings of longer names
_SURFACES_BY_LENGTH = sorted(_SURFACE_TO_CANONICAL, key=len, reverse=True)
 
 
def canonical_keys(text: str) -> set[str]:
    """For filtering and aggregation: which products does this text mention?"""
    tl = text.lower()
    found, consumed = set(), tl
    for surface in _SURFACES_BY_LENGTH:
        if surface in consumed:
            found.add(_SURFACE_TO_CANONICAL[surface])
            # Mask what we matched so it cannot be counted twice
            consumed = consumed.replace(surface, " " * len(surface))
    return found
 
 
def expand_query(q: str) -> str:
    """For search: append every other spelling of any product named in the query."""
    ql = q.lower()
    extra = []
    for alias in ALIASES:
        if any(s in ql for s in alias.surfaces):
            extra += [s for s in alias.surfaces if s not in ql]
    return q if not extra else q + " " + " ".join(dict.fromkeys(extra))
 
 
def stale_aliases(today: date, months: int = 18) -> list[ProductAlias]:
    """Aliases are trivial to add and hard to retire.
    Let the machine remember which ones are due for review."""
    out = []
    for a in ALIASES:
        if a.renamed_on and (today - a.renamed_on).days > months * 30:
            out.append(a)
    return out
 
 
if __name__ == "__main__":
    doc = "Vertex AI quota overages return 429. Retry with exponential backoff."
    print(canonical_keys(doc))
    # {'google.ai.platform'}  <- legacy-name doc still resolves
 
    print(canonical_keys("Gemini Enterprise Agent Platform skills are defined in YAML"))
    # {'google.ai.platform'}  <- new name lands on the same key
 
    print(expand_query("Gemini Enterprise Agent Platform IAM roles"))
    # returns the query with legacy spellings appended
 
    print(stale_aliases(date(2026, 7, 31)))
    # []  <- nothing due for review yet

The _SURFACES_BY_LENGTH ordering is where I tripped. I reasoned that "vertex ai" is not a substring of "gemini enterprise agent platform", so dictionary order would be fine. It broke from the other side: the moment I added the short alias aiplatform, it started colliding with longer names and double-counting. The two masking lines are what fixed it.

Operating This Over Time

Aliases accumulate. Left alone, the surface set grows until it starts dragging in unrelated documents.

That is what renamed_on is for. Anything more than 18 months past its rename gets surfaced as a review candidate — a human still makes the call, but the machine handles the remembering. There is nothing principled about 18 months. My apps take a little over a year for users to migrate off old versions, so I padded that.

Gemini naming is in motion right now. Pretraining for Gemini 4 has been announced, and the 3.5 and 3.6 model families are running side by side. Reading model IDs from configuration is well-trodden advice. The places that aggregate on the string in that config file are usually outside the blast radius anyone checks.

Names and availability keep shifting, so verify the rename details against the Gemini Enterprise release notes rather than this article.

What to Do Today

Start with one command against your own repository.

# Find hardcoded product names in comparisons, filters and aggregations
grep -rn --include='*.py' --include='*.ts' --include='*.sql' \
  -e '"Vertex AI"' -e "'Vertex AI'" -e 'Vertex AI' . \
  | grep -Ev '^\s*#|^\s*//' \
  | grep -E '==|in |LIKE|GROUP BY|filter|includes'

Of the lines that come back, fix the ones that fail without raising first. Search ranking can wait. What this measurement showed me is that the silent shortfall deserves attention before the visible degradation.

If you run the same benchmark, do not stop at 30 documents. Document frequency shifts with corpus size, and so does how much discriminating power a product name carries. Every number here holds only under "30 docs, 2-grams, BM25".

Measure before you reindex. That ordering is the part I will keep.

Thank You for Reading

Gemini Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.