When to Hand Conversation State to the Server: previous_interaction_id vs. Pruning History Yourself

Getting back to a design you can prune: a thin hybrid wrapper

So should you always hold history yourself? That is the other extreme. For short, straightforward conversations, the nimbleness of previous_interaction_id pays off. A design that resends all history every turn keeps paying its own cost in traffic and assembly.

So I settled on a hybrid: lean on server-side state by default, and only on the turn where a heavy blob arrives, break the chain and reassemble by hand. The deciding axis is a single question — "is this turn's output worth dragging along in full through every later turn?"

Below is a thin wrapper that seals that decision in one place. The caller only passes a conversation ID and never has to think about whether the chain is continued or restarted. Method and parameter names may vary by how the API is offered, so confirm the actual call against the official docs. What I want to show is the layer above that: where to put the pruning decision.

import re
from dataclasses import dataclass, field
 
# If a single turn's tool output exceeds this estimated token count,
# don't keep it on the chain — fold it into a summary instead.
HEAVY_TURN_TOKEN_THRESHOLD = 4000
 
 
def estimate_tokens(text: str) -> int:
    """Rough token estimate for mixed Japanese/English text.
    Not a real tokenizer; close enough for a threshold decision.
    ASCII counts as ~4 chars/token, other (mainly Japanese) as ~1.6 chars/token."""
    ascii_chars = len(re.findall(r"[\x00-\x7F]", text))
    other_chars = len(text) - ascii_chars
    return int(ascii_chars / 4 + other_chars / 1.6)
 
 
@dataclass
class Conversation:
    """Per-conversation state. If server_interaction_id is alive, continue the chain.
    If the chain was restarted, accumulate short client-side summaries in carryover."""
    server_interaction_id: str | None = None
    carryover: list[str] = field(default_factory=list)  # short summaries kept client-side
 
 
def summarize_heavy_output(raw: str) -> str:
    """Fold a heavy output down to the minimum worth dragging into later turns.
    In practice, put a single lightweight-model call or rule-based extraction here.
    Keep only 'what was decided'; discard the original."""
    head = raw.strip().splitlines()[0][:80] if raw.strip() else ""
    return f"[summary of previous tool output] {head} … full text discarded"
 
 
def send_turn(conv: Conversation, user_text: str, tool_output: str | None,
              call_model) -> tuple[str, Conversation]:
    """Send one turn. call_model wraps the actual Interactions API call:
    it takes (prompt_parts, previous_interaction_id) and returns (reply_text, interaction_id)."""
 
    heavy = tool_output is not None and \
        estimate_tokens(tool_output) > HEAVY_TURN_TOKEN_THRESHOLD
 
    if heavy or conv.server_interaction_id is None:
        # Restart the chain: stack prior carryover in front by hand, and
        # swap a heavy tool output for a summary before the model ever sees it.
        parts: list[str] = list(conv.carryover)
        if tool_output is not None:
            folded = summarize_heavy_output(tool_output) if heavy else tool_output
            parts.append(folded)
            if heavy:
                conv.carryover.append(folded)  # only the summary is dragged forward
        parts.append(user_text)
        reply, new_id = call_model(prompt_parts=parts, previous_interaction_id=None)
    else:
        # Happy path: continue the server-side chain plainly. Client stays empty.
        parts = [tool_output, user_text] if tool_output else [user_text]
        reply, new_id = call_model(prompt_parts=parts,
                                   previous_interaction_id=conv.server_interaction_id)
 
    conv.server_interaction_id = new_id
    return reply, conv

The crux is only the branch in send_turn. Normally it continues the server-side chain, so the client stays light. Only on a turn with heavy output does it break the chain once, move the folded summary from summarize_heavy_output into carryover, and drag only that summary forward. The 40KB original ends its role the moment the model reads it on that turn. It is not carried into the next.

What running it taught me

Once it was actually running, a few judgments outside the code turned out to matter.

First, a conservative threshold is easier to live with. I started by cutting at around 8,000 tokens, but each restart adds one lightweight-model call for summarizing, so too many switches let that call cost pile up. Set it too high and the drag comes back. Dropping it to around 4,000 tokens — matching the feel of "fold any tool output longer than two or three Japanese paragraphs" — is where it stabilized.

Second, discarding the original reliably matters more than the quality of the summary. Try to make the summary careful and your instruction to the lightweight model swells, becoming a new cost source of its own. In practice, if a single line of "what was decided" remains, the conversation continues. The design of not dragging beat any elaborate summary.

Third, the response quality on restarted turns dropped less than I had feared. I had braced for the conversation to break when the chain is cut and context is severed, but as long as carryover holds the summary of settled facts, the model picks up the thread fine. If anything, with the unneeded original gone, responses sometimes leaned closer to the point. That was the opposite of what I expected.

The per-user cost that goes invisible once you hand it over

There is one more thing server-side state quietly takes: per-user token attribution.

Back when I assembled history myself, I could count the input tokens of each request. Whose conversation used how much fed straight into internal usage-based allocation. But once you leave it to previous_interaction_id, a turn's input tokens include "invisible context the server is holding," and the total can no longer be read from your outbound payload size.

Since July 6, execution logs for supported Interactions API calls have been viewable in the AI Studio dashboard. You can trace each execution step and its consumption, so that becomes your first source of truth. But dashboard logs come with retention and sampling limits, so they cannot themselves be the authoritative record for a billing audit.

So I went with two ledgers. Every turn, I add the token usage the response reports (a usage-style field) to my own counter as conversation ID → cumulative tokens, and treat that as authoritative. The dashboard logs are relegated to a reconciliation target. With this in place, I can state which conversations were heavy each month in my own numbers, and a spike like the one above no longer goes unnoticed until the following month.

Where it lands, by situation

Let me fold all of this into a quick decision reference.

The point, I think, is not to assume previous_interaction_id is always the cheap choice. It is cheap only while the context you drag stays light. In conversations where a heavy blob mixes in, keeping a design you can prune in your own hands is exactly what separates one order of magnitude of the bill from another.

The next step

Start by pulling the cumulative input tokens per conversation ID from your own conversation logs. If even one conversation stands out, it almost always hides a "heavy blob dragged along after its job was done on the first call." Once you find it, drop in just the send_turn branch from this article first. Beginning the threshold around 4,000 tokens and adjusting as you watch the switch frequency is plenty.

I am still tuning that threshold per conversation type myself. If you are wrestling with the cost of your own automation in the same way, I hope this is of some use. Thank you for reading.