When Gemini's executed result and its prose disagree on a number — a gate that trusts only code_execution_result

Extracting the executed result in a structured way

output is stdout as a string, so you cannot treat it as a number as-is. Grabbing digits with a naive regex will catch some intermediate value instead. The robust move is to have the model print the final value in a machine-readable form, fixing the output format in the prompt.

PROMPT_SUFFIX = (
    "At the very end of your computation, print only the final result on a "
    "single line in this exact format, and use this format nowhere else: "
    "RESULT_JSON={\"weighted_ecpm\": <value>}"
)
 
response = client.models.generate_content(
    model="gemini-2.5-pro",
    contents=user_prompt + "\n" + PROMPT_SUFFIX,
    config=types.GenerateContentConfig(
        tools=[types.Tool(code_execution=types.ToolCodeExecution())],
    ),
)
 
import json, re
 
def extract_executed_value(parts, key):
    """Scan code_execution_result stdout for the RESULT_JSON line and
    return the value for the given key, or None if not found."""
    marker = re.compile(r'RESULT_JSON=(\{.*\})')
    found = None
    for part in parts:
        if part.code_execution_result is None:
            continue
        for line in part.code_execution_result.output.splitlines():
            m = marker.search(line)
            if m:
                try:
                    found = json.loads(m.group(1)).get(key)
                except json.JSONDecodeError:
                    continue
    return found  # keep the last RESULT_JSON if executed more than once
 
executed = extract_executed_value(response.candidates[0].content.parts, "weighted_ecpm")
print("executed final value:", executed)
# executed final value: 288.42105263157896

The value extract_executed_value returns is the only number allowed into your business logic. The prose's "about 288.4 yen" is display copy; it never drives a decision. In my own reporting at Dolice, once I drew this line between the primary value and the display value, the mysteries of mismatched numbers simply stopped.

Classifying OUTCOME and reacting to it

code_execution_result carries an outcome telling you whether the run succeeded. Read output without checking it and you hit a nasty bug: an empty output from a failed run gets misread as "the result is zero."

Fold this into a function that only takes values from a successful run.

def resolve_execution(parts, key):
    """Take the final value only from a successful run. Return failures
    with a reason instead of silently swallowing them."""
    marker = re.compile(r'RESULT_JSON=(\{.*\})')
    ok_value = None
    failures = []
    for part in parts:
        r = part.code_execution_result
        if r is None:
            continue
        outcome = getattr(r.outcome, "name", str(r.outcome))
        if outcome != "OUTCOME_OK":
            failures.append((outcome, r.output[:200]))
            continue
        for line in r.output.splitlines():
            m = marker.search(line)
            if m:
                try:
                    ok_value = json.loads(m.group(1)).get(key)
                except json.JSONDecodeError:
                    pass
    if ok_value is None:
        return {"status": "no_value", "failures": failures}
    return {"status": "ok", "value": ok_value, "failures": failures}
 
result = resolve_execution(response.candidates[0].content.parts, "weighted_ecpm")
print(result)
# {'status': 'ok', 'value': 288.42105263157896, 'failures': []}

Returning failures rather than discarding them matters. In production, a run that succeeded but failed once along the way is a sign of an unstable prompt, so I log it and revisit it later. For triaging empty responses in general, the approach in reverse-diagnosing empty Gemini responses from finish_reason transfers directly.

The gate: reconciling executed result against prose

This is the heart of it. Holding the executed value as truth, you mechanically check whether the model's prose contradicts that truth. If it does, you do not surface that response — you reject it. Prose that disagrees is a signal the model did not correctly understand its own result, and you should not show it to a reader.

def numbers_in_text(text):
    """Return the set of numbers (commas and decimals) found in text."""
    raw = re.findall(r'-?\d[\d,]*\.?\d*', text)
    vals = set()
    for r in raw:
        try:
            vals.add(float(r.replace(",", "")))
        except ValueError:
            pass
    return vals
 
def verify_prose(executed_value, prose_text, rel_tol=0.01):
    """Check that the prose contains a number matching the executed
    result within 1% relative error. If not, treat it as a contradiction."""
    for v in numbers_in_text(prose_text):
        if executed_value == 0:
            if v == 0:
                return True
        elif abs(v - executed_value) / abs(executed_value) <= rel_tol:
            return True
    return False
 
executed_value = result["value"]
prose_text = "".join(prose)
 
if verify_prose(executed_value, prose_text):
    final_answer = prose_text        # prose is consistent with the result
    trusted_number = executed_value  # the decision number is always the executed one
else:
    # prose is off -> replace what gets displayed
    final_answer = f"The weighted-average eCPM is {executed_value:.1f} yen (auto-formatted)."
    trusted_number = executed_value
    print("WARNING prose mismatch:", prose_text[:80])
 
print("display:", final_answer)
print("decision value:", trusted_number)

The relative-error check exists so the prose can round to "288.4" without being rejected. It forgives a dropped decimal but rejects a genuinely different number. In my AdMob case the prose contained only 312.4, with no value within 1% of the executed 287.9, so the gate caught it cleanly.

Treating the primary datum as the single source of truth and deriving everything else — display, explanation — from it pays off well beyond Code Execution. If you want the reasoning behind that design, the reliability chapter of Designing Data-Intensive Applications (Martin Kleppmann) is a useful lens on where to place your source of truth.

Traps you will hit in production

Once you build this, a few quiet mistakes are easy to make.

Mixing up multiple execution-result parts. When the model iterates, it may compute once, reconsider, and compute again. Grab the last output naively and it may not be the "computation for the final answer." That is exactly why you pin the final value with an explicit marker like RESULT_JSON= and break the dependence on ordering.

Expecting a value that was never printed to stdout. The sandbox returns results through standard output. You cannot read the value of a variable you never printed. Writing with a REPL's "last expression is returned" instinct will come up empty, so always print the final value.

Over-sensitive matching from floating point. When aggregation involves division, the rounding position differs between the executed result and the prose. Exact matching will reject everything, so relative error is the practical choice. Conversely, if rounding rules are fixed — as with currency — round on the execution side before writing it into RESULT_JSON so the display and the primary value line up more easily.

Overlooking cost. Code Execution consumes tokens for the code it runs. If you execute on every request for verification, recording tokens per request pays off; I collected that in recording production cost with Gemini's usageMetadata. If you want the basics of the feature itself first, getting Gemini to write and run code with the Code Execution API is a good on-ramp.

Wrapping up

If you use Code Execution for numeric aggregation, change one thing today: take the number that drives decisions from code_execution_result.output, not from the model's prose. The display sentence can stay as it is, but the value you pass to business logic flows through the executed result as primary data. Draw that single line and the class of accidents where a transcription hallucination trips you up nearly disappears.

Automating reporting taught me that "it's handy, so delegate it" and "verify the result" are two different jobs. Slipping in one small gate that keeps the executed result and the prose apart changes how much you can trust the automation. I am still finding my way here, but I hope it helps anyone else handing numbers to an AI.