Your Tool Results Are Quietly Eating the Conversation — Handle Passing to Keep Gemini Function Calling Contexts Lean

Implementation 1: a Token-Budgeted Compactor — Stop Sending Everything First

The first move is a layer that shrinks results to a budget before they go into functionResponse.

# compactor.py — shrink tool results to a token budget before functionResponse
# Solves: compounding input growth from re-sending fields nobody uses
import json
from google import genai
 
client = genai.Client()
MODEL = "gemini-flash-latest"
 
def count(text: str) -> int:
    return client.models.count_tokens(model=MODEL, contents=text).total_tokens
 
def compact_tool_result(rows: list[dict], keep_fields: list[str],
                        budget_tokens: int = 6000) -> dict:
    """Keep only priority fields; trim rows if still over budget.
    Always tell the model that trimming happened."""
    slim = [{k: r.get(k) for k in keep_fields} for r in rows]
    kept = len(slim)
    while kept > 1 and count(json.dumps(slim[:kept], ensure_ascii=False)) > budget_tokens:
        kept = int(kept * 0.8)  # shave 20% at a time until under budget
    return {
        "rows": slim[:kept],
        "total_rows": len(rows),
        "returned_rows": kept,
        "truncated": kept < len(rows),
        "note": "If truncated=true and you need all rows, call fetch_more",
    }
 
# Usage: keep only text and rating for reviews
# compacted = compact_tool_result(reviews, keep_fields=["text", "rating"])
# → 312 rows / 48,200 tokens fits within 6,000 tokens

Two things matter here. First, always disclose the trim via the truncated flag and counts. Trim silently and the model will aggregate under the false belief that "there are only 287 reviews," producing a quietly wrong report. Second, cut fields before rows. Dropping rows first biases the sample; fields irrelevant to the task can go safely.

Implementation 2: Handle Passing — Keep the Payload Local, Give the Model a Table of Contents

When compaction isn't enough, flip the model of ownership: store the full result locally, and return only a count, a schema, a short digest, and a reference ID. The model pulls details through a second tool only when it actually needs them.

# handle_passing.py — pass tool results by reference
# Solves: keeping huge payloads out of history, fetching fragments on demand
import uuid
 
class ResultStore:
    """A store scoped to one run. Cross-run references fail loudly."""
    def __init__(self):
        self._data: dict[str, list[dict]] = {}
 
    def put(self, rows: list[dict]) -> str:
        handle = f"res_{uuid.uuid4().hex[:8]}"
        self._data[handle] = rows
        return handle
 
    def slice(self, handle: str, offset: int = 0, limit: int = 40) -> dict:
        if handle not in self._data:
            return {"error": f"handle {handle} does not exist in this run"}
        rows = self._data[handle][offset:offset + limit]
        return {"rows": rows, "offset": offset,
                "total": len(self._data[handle])}
 
store = ResultStore()
 
def fetch_reviews(store_id: str) -> dict:
    """Only this return value reaches the model. Full data goes to ResultStore."""
    rows = load_reviews_from_db(store_id)  # your data access here
    handle = store.put(rows)
    return {
        "handle": handle,
        "total_rows": len(rows),
        "schema": ["text", "rating", "date", "lang"],
        "digest": summarize_locally(rows),  # light stats only, e.g. counts per star
        "note": "Call fetch_rows(handle, offset, limit) if you need raw rows",
    }
 
def fetch_rows(handle: str, offset: int, limit: int) -> dict:
    """The second tool, called only when the model asks for detail"""
    return store.slice(handle, offset, min(limit, 40))

Capping limit (40 rows here) is the operational trick. Without a cap, the model will politely ask for everything, and handle passing degenerates back into full-payload passing.

Tasks that genuinely need to touch every row — classification, say — shouldn't ride the conversation loop at all. Feed the handle's contents to a local batch process (small per-row calls or the Batch API) instead. The agent conversation keeps the judgment; the work happens outside the history.

Folding Old Turns — in an Order That Doesn't Break Thought Signatures

If you're retrofitting a running agent, cleaning up the history itself also pays off: before assembling the next request, replace the body of spent functionResponse parts with a stub like {"status": "processed", "handle": "res_xxxx"}.

One caution for Gemini 3-family models: thought signatures, used to validate multi-turn integrity, must not be deleted or altered. Only fold the functionResponse bodies you authored; leave model-produced parts, signatures included, untouched. I've written up the details in keeping thought signatures intact across multi-turn Function Calling.

And if what you actually want to compress is a long user conversation, that's a different problem with different constraints — covered in compressing Gemini chat history with rolling summaries. This article is strictly about the residency of machine-made payloads.

Measured Results: Input Tokens Down ~8x, p95 from 34s to 9s

After two weeks of running the review agent with handle passing plus stub folding:

To be honest, it isn't all upside. Those fetch_rows calls add an average of 1.3 extra turns per run, so the loop is longer. But eliminating the re-sends dominates: total time and cost both dropped sharply. The richer your local digest, the fewer detail fetches you'll see — I treat that as a dial to tune in production rather than something to get right up front.

Pitfalls — Compact Too Hard and the Model Comes Back for More

Three I actually hit:

Over-tightening the budget creates a re-fetch loop. At a 2,000-token budget, the model called fetch_more four times in a row, and turn count went up instead of down. Start the budget at "all rows of the fields the task needs," and only tighten while measuring.

Vague handle lifetimes. My store wasn't persisted across process restarts, so a retried run dereferenced a stale handle and got nothing. Making ResultStore fail loudly ("does not exist in this run") let the model recover naturally by calling fetch_reviews again.

Breaking correspondence under parallel function calls. When multiple functionCall parts arrive in one turn, your functionResponse parts must match them in order and name. My stub-folding once disturbed that pairing and earned an INVALID_ARGUMENT. Fold bodies only; never touch part ordering.

If you also want a hard ceiling on spend before submitting work, the budget gate in stopping cost overruns before batch submission with countTokens composes cleanly with everything above.

Wrapping Up — Chart One Run First

Before debating designs, add the one-line log_turn_tokens to your existing loop and look at a single run's trajectory. If input jumps in a staircase from turn two onward, the compaction and handle passing here will apply directly. If it doesn't, your loop is still healthy — and confirming that takes five minutes, which is the safest possible place to start.