When to Hand Conversation State to the Server: previous_interaction_id vs. Pruning History Yourself

Getting back to a design you can prune: a thin hybrid wrapper

So should you always hold history yourself? That is the other extreme. For short, straightforward conversations, the nimbleness of previous_interaction_id pays off. A design that resends all history every turn keeps paying its own cost in traffic and assembly.

So I settled on a hybrid: lean on server-side state by default, and only on the turn where a heavy blob arrives, break the chain and reassemble by hand. The deciding axis is a single question — "is this turn's output worth dragging along in full through every later turn?"

Below is a thin wrapper that seals that decision in one place. The caller only passes a conversation ID and never has to think about whether the chain is continued or restarted. Method and parameter names may vary by how the API is offered, so confirm the actual call against the official docs. What I want to show is the layer above that: where to put the pruning decision.

import re
from dataclasses import dataclass, field
 
# If a single turn's tool output exceeds this estimated token count,
# don't keep it on the chain — fold it into a summary instead.
HEAVY_TURN_TOKEN_THRESHOLD = 4000
 
 
def estimate_tokens(text: str) -> int:
    """Rough token estimate for mixed Japanese/English text.
    Not a real tokenizer; close enough for a threshold decision.
    ASCII counts as ~4 chars/token, other (mainly Japanese) as ~1.6 chars/token."""
    ascii_chars = len(re.findall(r"[\x00-\x7F]", text))
    other_chars = len(text) - ascii_chars
    return int(ascii_chars / 4 + other_chars / 1.6)
 
 
@dataclass
class Conversation:
    """Per-conversation state. If server_interaction_id is alive, continue the chain.
    If the chain was restarted, accumulate short client-side summaries in carryover."""
    server_interaction_id: str | None = None
    carryover: list[str] = field(default_factory=list)  # short summaries kept client-side
 
 
def summarize_heavy_output(raw: str) -> str:
    """Fold a heavy output down to the minimum worth dragging into later turns.
    In practice, put a single lightweight-model call or rule-based extraction here.
    Keep only 'what was decided'; discard the original."""
    head = raw.strip().splitlines()[0][:80] if raw.strip() else ""
    return f"[summary of previous tool output] {head} … full text discarded"
 
 
def send_turn(conv: Conversation, user_text: str, tool_output: str | None,
              call_model, tool_roundtrip_open: bool = False
              ) -> tuple[str, Conversation]:
    """Send one turn. call_model wraps the actual Interactions API call:
    it takes (prompt_parts, previous_interaction_id) and returns (reply_text, interaction_id).
    tool_roundtrip_open=True means the model returned a function call whose result has not
    been sent back yet. Break the chain there and the call and its result drift apart."""
 
    heavy = tool_output is not None and \
        estimate_tokens(tool_output) > HEAVY_TURN_TOKEN_THRESHOLD
 
    # Only restart once the tool round trip has closed
    if (heavy and not tool_roundtrip_open) or conv.server_interaction_id is None:
        # Restart the chain: stack prior carryover in front by hand, and
        # swap a heavy tool output for a summary before the model ever sees it.
        parts: list[str] = list(conv.carryover)
        if tool_output is not None:
            folded = summarize_heavy_output(tool_output) if heavy else tool_output
            parts.append(folded)
            if heavy:
                conv.carryover.append(folded)  # only the summary is dragged forward
        parts.append(user_text)
        reply, new_id = call_model(prompt_parts=parts, previous_interaction_id=None)
    else:
        # Happy path: continue the server-side chain plainly. Client stays empty.
        parts = [tool_output, user_text] if tool_output else [user_text]
        reply, new_id = call_model(prompt_parts=parts,
                                   previous_interaction_id=conv.server_interaction_id)
 
    conv.server_interaction_id = new_id
    return reply, conv

The crux is only the branch in send_turn. Normally it continues the server-side chain, so the client stays light. Only on a turn with heavy output does it break the chain once, move the folded summary from summarize_heavy_output into carryover, and drag only that summary forward. The 40KB original ends its role the moment the model reads it on that turn. It is not carried into the next.

What running it taught me

Once it was actually running, a few judgments outside the code turned out to matter.

First, a conservative threshold is easier to live with. I started by cutting at around 8,000 tokens, but each restart adds one lightweight-model call for summarizing, so too many switches let that call cost pile up. Set it too high and the drag comes back. Dropping it to around 4,000 tokens — matching the feel of "fold any tool output longer than two or three Japanese paragraphs" — is where it stabilized.

Second, discarding the original reliably matters more than the quality of the summary. Try to make the summary careful and your instruction to the lightweight model swells, becoming a new cost source of its own. In practice, if a single line of "what was decided" remains, the conversation continues. The design of not dragging beat any elaborate summary.

Third, the response quality on restarted turns dropped less than I had feared. I had braced for the conversation to break when the chain is cut and context is severed, but as long as carryover holds the summary of settled facts, the model picks up the thread fine. If anything, with the unneeded original gone, responses sometimes leaned closer to the point. That was the opposite of what I expected.

The per-user cost that goes invisible once you hand it over

There is one more thing server-side state quietly takes: per-user token attribution.

Back when I assembled history myself, I could count the input tokens of each request. Whose conversation used how much fed straight into internal usage-based allocation. But once you leave it to previous_interaction_id, a turn's input tokens include "invisible context the server is holding," and the total can no longer be read from your outbound payload size.

Since July 6, 2026, execution logs for supported Interactions API calls have been viewable in the AI Studio dashboard. You can trace each execution step and its consumption, so that becomes your first source of truth. But dashboard logs come with retention and sampling limits, so they cannot themselves be the authoritative record for a billing audit.

So I went with two ledgers. Every turn, I add the token usage the response reports (a usage-style field) to my own counter as conversation ID → cumulative tokens, and treat that as authoritative. The dashboard logs are relegated to a reconciliation target. With this in place, I can state which conversations were heavy each month in my own numbers, and a spike like the one above no longer goes unnoticed until the following month.

Catching the drag in numbers: a drag ratio per conversation ID

Once the two ledgers were in place, I wanted to reach for anomalies without waiting for the month to close. Noticing on last month's invoice means noticing after the money is already spent.

What I run every morning now is a small script that reads the JSONL from my own counter and reports a drag ratio per conversation. It treats the first turn's input tokens as "the state with no drag" and asks how many times larger the final turn grew.

import json
from collections import defaultdict
 
 
def load_ledger(path: str) -> dict[str, list[int]]:
    """Fold the counter's JSONL into per-conversation input-token series.
    A line looks roughly like {"conversation_id": "...", "turn": 3, "input_tokens": 21840}."""
    per_conv: dict[str, list[int]] = defaultdict(list)
    with open(path, encoding="utf-8") as f:
        for line in f:
            if not line.strip():
                continue
            rec = json.loads(line)
            per_conv[rec["conversation_id"]].append(
                (int(rec.get("turn", 0)), int(rec["input_tokens"]))
            )
    # Re-sort by turn and keep only the token series
    return {cid: [t for _, t in sorted(turns)] for cid, turns in per_conv.items()}
 
 
def drag_report(per_conv: dict[str, list[int]], min_turns: int = 4) -> list[dict]:
    """Rank conversations where drag is biting, by tokens eaten by accumulated context."""
    rows = []
    for cid, inputs in per_conv.items():
        if len(inputs) < min_turns:
            continue  # short conversations end before drag accumulates
        base = max(inputs[0], 1)
        # Difference from "every turn as light as the first" = what context ate
        carried = sum(inputs) - base * len(inputs)
        rows.append({
            "conversation_id": cid,
            "turns": len(inputs),
            "base_in": base,
            "last_in": inputs[-1],
            "drag_ratio": round(inputs[-1] / base, 1),
            "carried_tokens": max(carried, 0),
        })
    rows.sort(key=lambda r: r["carried_tokens"], reverse=True)
    return rows
 
 
if __name__ == "__main__":
    report = drag_report(load_ledger("usage_ledger.jsonl"))
    for row in report[:10]:
        print("{conversation_id}\tturns={turns}\tx{drag_ratio}\t"
              "carried={carried_tokens}".format(**row))

Sorting by carried_tokens rather than drag_ratio is deliberate. Rank by ratio alone and light conversations that only ever held a few hundred tokens float to the top with wild multipliers, and you end up chasing conversations that never touched the bill. When you want what you are actually paying, absolute volume is the plainer axis.

Running a month of my own data through it, conversations with a drag_ratio above 6 did not even reach one in ten. Yet they accounted for more than half of all carried_tokens. Drag does not spread thinly; it clumps into a handful of conversations. So before tuning the threshold, go look at those top few.

One more thing worth recording is a failure I hit while restarting. The model had returned a function call and I had not yet sent its result back — the tool round trip was still open — and cutting the chain there separated the call from its result, leaving the response out of step. That is why tool_roundtrip_open exists in the wrapper above. Weight alone is not permission to cut; the round trip has to close first.

Where it lands, by situation

Let me fold all of this into a quick decision reference.

The point, I think, is not to assume previous_interaction_id is always the cheap choice. It is cheap only while the context you drag stays light. In conversations where a heavy blob mixes in, keeping a design you can prune in your own hands is exactly what separates one order of magnitude of the bill from another.

The next step

Start by pulling the cumulative input tokens per conversation ID from your own conversation logs. If even one conversation stands out, it almost always hides a "heavy blob dragged along after its job was done on the first call." Once you find it, drop in just the send_turn branch from this article first. Beginning the threshold around 4,000 tokens and adjusting as you watch the switch frequency is plenty.

I am still tuning that threshold per conversation type myself. If you are wrestling with the cost of your own automation in the same way, I hope this is of some use. Thank you for reading.