Don't make Gemini judge your AdMob report — confine structured output to extraction

Put judgment in code — an auditable decision function

Against the extracted, typed data, judgment happens in an ordinary Python function. This is the crux of auditability.

from dataclasses import dataclass
 
@dataclass
class FloorDecision:
    group_id: str
    current: float
    proposed: float
    action: str       # "raise" / "lower" / "hold"
    reason: str       # keep the basis as a string (for audit)
 
def decide_floor_ios(row) -> FloorDecision:
    ratio = row.current_floor_usd / row.ecpm_usd if row.ecpm_usd else 0
    target = round(row.ecpm_usd * 0.55 / 0.5) * 0.5    # 55% basis, $0.50 steps
 
    if ratio > 0.65:
        return FloorDecision(row.group_id, row.current_floor_usd,
                             max(target, 0.5), "lower",
                             f"ratio {ratio:.0%} > 65%")
    if ratio < 0.40 and row.match_rate > 0.95:
        return FloorDecision(row.group_id, row.current_floor_usd,
                             max(target, 0.5), "raise",
                             f"ratio {ratio:.0%} < 40% & mr {row.match_rate:.0%} > 95%")
    return FloorDecision(row.group_id, row.current_floor_usd,
                         row.current_floor_usd, "hold",
                         f"ratio {ratio:.0%} in 40-65% band")
 
decisions = [decide_floor_ios(r) for r in report.rows]

Keeping the basis as a string in the reason field is the point. If you are later asked "why was this group a hold," a mechanical basis like ratio 64% in 40-65% band comes out instantly. Had you let an LLM do it, you would never get that one line. When you want to change the logic, you just write a test and fix the function — no need to "grow" a prompt.

How to guarantee the extraction is correct

Even with judgment shifted into code, it is all for nothing if the extraction is wrong. So pass the extracted result through light sanity checks on the code side.

def sanity_check(row) -> list[str]:
    issues = []
    if not (0.0 <= row.match_rate <= 1.0):
        issues.append(f"{row.group_id}: abnormal match_rate {row.match_rate}")
    if row.ecpm_usd < 0 or row.ecpm_usd > 200:
        issues.append(f"{row.group_id}: abnormal eCPM {row.ecpm_usd}")
    if row.current_floor_usd < 0:
        issues.append(f"{row.group_id}: negative floor")
    return issues

If the match rate exceeds 1.0, Gemini likely put "47.53%" in as 47.53. An abnormal eCPM suggests a misread currency symbol. Extraction by the LLM, validation by code — the same division as the previous section: the LLM for fuzzy input, code for deterministic verification. Operational filters, like excluding groups with fewer than 20 impressions from judgment, are also made explicit here as code.

Reconcile whether the extracted rows cover every group

Alongside per-row sanity checks, the one I added after getting burned in real operation is reconciling the row count itself. Gemini's extraction fails not only on value correctness but on dropping groups. If the tail of the report is cut off, or two similarly named groups get merged into one row, the 42 groups that should exist come back as 41 rows. Every value looks plausible, so the previous section's sanity check waves it through.

So after extraction, reconcile against "the set of group IDs I expect."

EXPECTED_GROUP_IDS = {...}  # hold the live group IDs as a constant
 
def reconcile_rows(report) -> list[str]:
    got = {r.group_id for r in report.rows}
    missing = EXPECTED_GROUP_IDS - got
    unexpected = got - EXPECTED_GROUP_IDS
    issues = []
    if missing:
        issues.append(f"dropped: {sorted(missing)}")
    if unexpected:
        issues.append(f"unknown group_id: {sorted(unexpected)}")
    if len(report.rows) != len({r.group_id for r in report.rows}):
        issues.append("duplicate group_id (one group may be split into two rows)")
    return issues

missing is a drop, unexpected is the trace of a notation variant mistaken for a different group, and a duplicate is the sign of one group split into two rows. Group composition barely changes month to month, so simply holding the known set as a constant catches almost any anomaly in the extraction "count." Only with both stages — value verification (previous section) and count reconciliation (this section) — is the extracted result ready to hand to judgment.

Leave the write to the human — designing the pipeline's terminus

Even after extraction and judgment, the final move of typing the floor into AdMob's edit screen I leave to the human. AdMob's edit screen has dynamic button IDs, and auto-entry is prone to misfiring into the wrong field. The pipeline's output stops at "a list of proposed values and bases per group"; the human reads it and enters them. After entry, the actual value on the edit screen is read back by the same mechanism as the extraction and reconciled against the proposed value.

This terminus design yields a pipeline where each step's responsibility is cleanly separated: "Gemini extracts → code judges → human writes → code reconciles the actual value." The LLM is involved only in the first extraction, while the revenue-critical judgment and the irreversible write are held by code and the human respectively. This is the basic form I recommend for automation that handles revenue data.

Where this design applies

Finally, this "separate extraction from judgment" design applies beyond AdMob floor decisions. Any task that meets the condition — the decision rules can be written deterministically, while the input data is messy — becomes a fast, auditable automation when you extract with structured output and place judgment in code. Tasks where the judgment itself is fuzzy, like review sentiment classification, lean to the LLM; judgments decided by thresholds or arithmetic lean to code. Deciding this line first is, I think, the starting point of design when embedding an LLM into operations.

Start by reviewing one prompt you are sending to an LLM and separating "is this extraction, or judgment?" Just moving the judgment part into code should sharply raise the reproducibility and auditability of the output.