A minimal pre-send guard for Gemini-drafted app review replies

I once started writing a reply to a one-star review late at night.

"It crashes every time I open settings." I knew the bug. Before I noticed, I had typed "we will fix this in the next update," and I stopped with my cursor on the send button. Nothing at that moment guaranteed the fix would make the next build.

Gemini can draft that reply in seconds. The faster the drafting gets, the less recoverable the send becomes. Store replies are public, and a promise stays readable until the next version actually ships.

So before making the drafting smarter, I built the part that stops a reply from going out. The code is short and unglamorous. But when I ran it, the thing that turned out to be wrong was the guard, not the draft — and that part is worth showing.

Decide whether to reply before writing anything

If you answer reviews in the order they arrive, you run out of time for the ones that matter. So the sorting happens before any text is generated.

Bucket	Condition	What happens next
`priority`	2 stars or fewer, with something resembling repro steps	Draft it, and read the draft yourself
`ack_only`	3 stars or fewer, no repro steps	Send a short acknowledgement
`no_reply`	4 stars or more	Don't spend a reply here

Skipping happy reviewers may look cold. But the people whose opinion actually moves when they read a reply are the ones currently stuck. That is where the limited time goes.

import re
 
REPRO = r"(when I|after I|every time|crash|freeze|won't open|can't save|doesn't load)"
 
def triage(stars: int, review: str) -> str:
    repro = re.search(REPRO, review, re.IGNORECASE)
    if stars >= 4:
        return "no_reply"        # don't spend a reply here
    if stars <= 2 and repro:
        return "priority"        # repro steps present, handle first
    return "ack_only"            # acknowledge and move on

The only signals here are the star rating and symptom words. Emotional intensity is deliberately ignored. A calmly written report of exact steps usually carries more fixable information than an angry one.

Receive the draft as structure, not prose

Ask Gemini for "a reply" and you get one continuous paragraph. Feeding that into a guard means re-parsing which sentence is an apology and which one is a commitment, every single time.

Splitting it at the source makes everything downstream easier.

from google import genai
from google.genai import types
 
client = genai.Client(api_key="YOUR_API_KEY")
 
REPLY_SCHEMA = {
    "type": "object",
    "properties": {
        "acknowledgement": {"type": "string"},   # we heard you
        "status": {"type": "string"},            # only what is known today
        "next_step": {"type": "string"},         # what we ask of the reviewer
    },
    "required": ["acknowledgement", "status"],
}
 
def draft(review: str, mode: str) -> dict:
    prompt = (
        "Draft a reply to this app store review. Stay polite and concise, "
        "and never state a fix date that has not been committed.\n"
        f"Bucket: {mode}\nReview: {review}"
    )
    res = client.models.generate_content(
        model="gemini-3.7-flash",
        contents=prompt,
        config=types.GenerateContentConfig(
            response_mime_type="application/json",
            response_schema=REPLY_SCHEMA,
        ),
    )
    return res.parsed

Keeping status separate is the point. "We are investigating the cause" is a fact; "we will fix it" is a commitment. Blend them into one paragraph and you can no longer tell them apart afterwards. If the schema itself gives you trouble, common Structured Output validation errors and how to resolve them collects the usual failure shapes.

And telling the model not to promise a date does not mean it never will. That is why the guard exists.

Three checks before send

Only three things need to be stopped: promises you cannot keep (A), contact details leaking into a public reply (B), and tone or length problems (C).

MAX_CHARS = 350  # my own readability limit, separate from any store limit
 
PROMISE = [r"次の(アップデート|バージョン)で", r"必ず(修正|対応)", r"すぐに(修正|対応)",
           r"\d+\s*(日|週間|月)以内に", r"来週(まで)?に", r"今月中に", r"予定です"]
# Japanese has an explicit polite register. Confirm it first, then suspect the plain form.
POLITE_END = re.compile(r"(です|でした|ます|ました|ません|ましょう|ください|ございます)。$")
PLAIN_END = re.compile(r"(だ|である|した|ない|いる|する|よ|ね)。$")
 
def mask(draft: str) -> str:
    """Blank out emails and URLs first, or the handle pattern fires twice on the same span"""
    d = re.sub(r"[\w.+-]+@[\w-]+\.[\w.]+", "\u3000", draft)
    return re.sub(r"https?://\S+", "\u3000", d)
 
def check(draft: str) -> list:
    hits = []
    for p in PROMISE:
        if re.search(p, draft):
            hits.append(("A_promise", p))
    for pattern, target in ((r"[\w.+-]+@[\w-]+\.[\w.]+", draft),
                            (r"https?://\S+", draft),
                            (r"@[A-Za-z0-9_]{3,}", mask(draft)),
                            (r"0\d{1,4}-\d{1,4}-\d{3,4}", mask(draft))):
        m = re.search(pattern, target)
        if m:
            hits.append(("B_contact", m.group(0)))
    for sentence in re.split(r"(?<=。)", draft):
        sentence = sentence.strip()
        if sentence and not POLITE_END.search(sentence) and PLAIN_END.search(sentence):
            hits.append(("C_plain", sentence))
            break
    if len(draft) > MAX_CHARS:
        hits.append(("C_length", f"{len(draft)} chars"))
    return hits

Check B exists because a public reply that routes someone to a support address leaves that address sitting in public forever. The instinct to help one person individually is right; the place to do it is not the store listing.

Running it exposed two bugs — in the guard

I prepared six review-and-draft pairs and ran the first version as written. The drafts are Japanese, since that is the locale I actually reply in, so check C evaluates the Japanese polite register. The output was this:

★  bucket     verdict  hits
1  priority   STOP     A_promise:次の(アップデート|バージョン)で
1  ack_only   STOP     B_contact:support@example.com / B_contact:@example
2  ack_only   OK       —
2  priority   STOP     C_plain:保存が完了しない事象を確認しました。
5  no_reply   —        reply slot not used
3  ack_only   STOP     C_plain:ご意見に感謝する。
 
6 drafts → 5 in scope / 1 passed / 4 blocked / 1 skipped

Two of those four blocks were the guard's own mistakes.

The first is on line two. B_contact fires twice for support@example.com, because the handle pattern picked up @example inside the email address. One span, counted twice.

The second one actually hurts. On line four, a perfectly polite sentence was blocked as plain-form, because the rule searched for a plain-form ending (した。) that also sits inside the polite ending (ました。). Ordering was the problem: the guard has to confirm the polite form first and only then suspect the plain one, not the other way around. The same trap exists in any language where the formal marker is a superset of the informal one.

Two fixes: mask emails and URLs before looking for handles, and evaluate the polite ending before the plain one. Re-running gives:

1  priority   STOP     A_promise:次の(アップデート|バージョン)で
1  ack_only   STOP     B_contact:support@example.com
2  ack_only   OK       —
2  priority   OK       —
5  no_reply   —        reply slot not used
3  ack_only   STOP     C_plain:ご意見に感謝する。
 
6 drafts → 5 in scope / 2 passed / 3 blocked / 1 skipped

Metric	Before	After
Blocked	4	3
False positives	2 (one double count, one register misread)	0
Passed	1	2

All three remaining blocks were correct: an uncommitted fix date, an address I did not want published, and a sentence that dropped out of the register I use with reviewers.

A guard that cries wolf gets ignored, and an ignored guard eventually gets deleted. Laying six cases out by hand while the numbers are still small is cheap insurance against that path.

What to delegate, and what to keep

Drafting and mechanical checking are automated on my side. Pressing send is not.

The reason is simple: a reply contains commitments. "We are investigating" reports a fact, but "we will fix this" agrees to a development plan. Only the person who knows what gets worked on next can make that call, and that has nothing to do with how well the sentence is written.

The same logic keeps judgement about the review itself in human hands. The buckets inform me; they do not decide for me. A reply written without reading the review reads exactly like one.

Once you are replying in several languages, other concerns appear — terminology consistency, pacing the volume, register per locale. I wrote up an actual multilingual run in building Google Play replies in eleven languages with Gemini, which is worth a look once your reply volume grows.

One thing to try next

Add a single phrase to the PROMISE list: one you have personally regretted sending. A generic banned-words list helps less than removing one of your own habits. Mine was "shortly."

It is a small guard. But having one stage in front of the send button means the version of you that writes at midnight gets checked by the version that reads at breakfast.