◈ API / SDK/2026-08-07Advanced

When Streaming Responses Quietly Lose Non-ASCII Text: Measuring the Byte Boundary and the Event Boundary Separately

I measured why Japanese text disappears from streaming Gemini responses using a local mock SSE server. Two separate layers were broken, one of them silently, and an English-only test suite caught neither.

Gemini API²⁵⁵ Streaming⁵SSEUTF-8Internationalization

✦ Premium Article

I was wiring Gemini's streaming output into a live text view for a wallpaper app, showing description copy as it was generated.

Everything looked fine. Characters appeared one by one, the paragraph filled in, the stream closed cleanly. On my machine it never failed.

What caught my attention was the ending, on a real device. The text was slightly short.

It still read naturally. The grammar was intact. But when I ran the same prompt without streaming and put the two responses side by side, the final sentence was missing entirely.

The error log was empty. The status was 200. Nothing had thrown.

Nothing appeared broken, and yet something had not arrived. That combination bothered me enough to stop guessing and start measuring, so I built a mock server locally.

Two separate layers turned out to be broken. One of them failed in exactly the opposite way from what I had assumed.

The measurement setup

Depending on a live service makes numbers move around. Instead I served a byte-for-byte imitation of Gemini's streamGenerateContent?alt=sse response shape from localhost.

Python 3.10.12 and Node.js v22.22.3
Payload: 8 SSE events, 1,406 bytes total. Of that, the Japanese body text is 85 characters — three bytes each, so 255 bytes, or 18.1% of the stream
A structurally identical English payload (1,315 bytes) for comparison

The important property is that the server controls exactly how many bytes it writes to the socket at a time. Real networks fragment responses through TCP segmentation and proxy buffering, and you can neither observe nor control that. So force it on the test side instead.

# server.py — a mock SSE server that writes a fixed number of bytes at a time
import socket, sys, time, json
 
JA_PARTS = [
    "画像分類パイプラインの", "移行を進めています。", "停止予定のモデルを",
    "洗い出したところ、", "参照箇所は思ったより", "広範囲に散っていました。",
    "特に設定ファイル側の", "既定値が見落とされがちです。",
]
 
def build(parts, model="gemini-3.6-flash"):
    """Assemble a body with the same shape as a Gemini SSE response."""
    out = []
    for i, p in enumerate(parts):
        obj = {
            "candidates": [{
                "content": {"parts": [{"text": p}], "role": "model"},
                "index": 0,
            }],
            "modelVersion": model,
        }
        if i == len(parts) - 1:
            obj["candidates"][0]["finishReason"] = "STOP"
            obj["usageMetadata"] = {
                "promptTokenCount": 48,
                "candidatesTokenCount": 96,
                "totalTokenCount": 144,
            }
        # ensure_ascii=False matters: emit real UTF-8, not \u escapes
        out.append("data: " + json.dumps(obj, ensure_ascii=False) + "\n\n")
    return "".join(out)
 
def serve(port, chunk):
    body = build(JA_PARTS).encode("utf-8")
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind(("127.0.0.1", port))
    s.listen(8)
    conn, _ = s.accept()
    with conn:
        conn.recv(65536)  # discard the request line
        conn.sendall((
            "HTTP/1.1 200 OK\r\n"
            "Content-Type: text/event-stream; charset=utf-8\r\n"
            "Cache-Control: no-cache\r\n"
            f"Content-Length: {len(body)}\r\n"
            "Connection: close\r\n\r\n"
        ).encode())
        # the point of the exercise: deliberately fragmented writes
        for i in range(0, len(body), chunk):
            conn.sendall(body[i:i + chunk])
            time.sleep(0.001)
    s.close()
 
if __name__ == "__main__":
    serve(int(sys.argv[1]), int(sys.argv[2]))

That time.sleep(0.001) is not decoration. Without it the kernel coalesces the writes in the send buffer, the receiver sees whatever size it likes, and the chunk parameter stops meaning anything. I left it out on the first pass, got identical results at every chunk size, and spent a while confused before working out why.

The byte boundary breaks only non-ASCII text

I started with the most common shape of mistake: decode each received chunk on its own with decode("utf-8", errors="replace").

Varying the server's write size, I counted how many U+FFFD replacement characters appeared in the reassembled body. Separately, I walked the payload bytes analytically to count how many boundaries landed in the middle of a multi-byte character.

Write size	Boundaries	Characters split	U+FFFD emitted	Share of the 85-char body corrupted
8 bytes	175	21	54	24.7%
12 bytes	117	20	40	23.5%
16 bytes	87	10	26	11.8%
24 bytes	58	9	18	10.6%
32 bytes	43	5	13	5.9%
48 bytes	29	4	8	4.7%
64 bytes	21	2	5	2.4%
128 bytes	10	1	2	1.2%
256 bytes	5	0	0	0%

Each split character produced either two or three replacement characters. A three-byte sequence cut as 2+1 yields two; cut as 1+2 it yields three. The measured counts fall exactly into those two buckets, which is a useful sanity check that the harness is measuring what it claims to.

Running the identical sweep against the English payload is what convinced me this was worth writing up.

Across all 40 conditions — five implementations times eight chunk sizes — zero corruption.

ASCII is single-byte, so no boundary can ever land inside a character. Japanese is three bytes, giving every character two positions where a cut is fatal. Emoji are four bytes, giving three.

# how many cut positions are fatal, per character width
s = "完了しました🎉"
b = s.encode()
for i in range(1, len(b)):
    joined = b[:i].decode("utf-8", "replace") + b[i:].decode("utf-8", "replace")
    n = joined.count("�")
    if n:
        print(f"split@{i:2} -> U+FFFD={n}  {joined!r}")
# split@ 1 -> U+FFFD=3  '���了しました🎉'
# split@19 -> U+FFFD=4  '完了しました����'

A four-byte emoji can produce four replacement characters from a single cut. If your product surfaces emoji in generated copy — and most consumer-facing ones do — this is not a localization-only concern. It is simply a non-ASCII concern, and emoji are more fragile than kanji.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦Corruption rates measured per chunk size (24.7% of body text at 8 bytes, 0% at 256 bytes) plus the complete mock SSE server you can run yourself

✦Why the same mistake raises an exception in Python but silently substitutes U+FFFD in Node.js, and the correct receive loop for both runtimes

✦The second boundary that survives a UTF-8 fix, where whole events vanish behind a 200 response, and the adversarial server to put in CI

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

The same mistake is loud in Python and silent in Node

This is where my expectations were furthest from the measurements.

I assumed that if text was being mangled, something would throw. In practice it depends entirely on which runtime you are in.

How the stream is consumed	Result	Detectable?
Python raw socket + `chunk.decode("utf-8")`	`UnicodeDecodeError` raised	Yes
Python raw socket + `decode("utf-8", errors="replace")`	No exception, 26 U+FFFD	No
Node.js `new TextDecoder().decode(value)`	No exception, 26 U+FFFD	No
Node.js decoder reused with the stream option	Exact match at every chunk size	—
Python `codecs.getincrementaldecoder("utf-8")()`	Exact match at every chunk size	—

Without the stream option, TextDecoder treats the bytes you hand it as a complete, self-contained unit. Any trailing partial sequence gets replaced. That is the specified behavior, not a defect.

The trouble is that wrapping the call in try catches nothing. No log line is produced. The JSON stays structurally valid, because U+FFFD is a perfectly legal character inside a JSON string, so JSON.parse succeeds too. In my runs the parsed event count stayed at 8 in every single condition.

In other words, every signal you monitor stays green while the rendered output is wrong.

The stream option tells the decoder that more bytes are coming. It holds any incomplete sequence internally and joins it with the next chunk before producing characters.

// Works in Node.js and in browsers. The decoder instance must be reused.
const res = await fetch(url, { headers: { Accept: "text/event-stream" } });
const reader = res.body.getReader();
const decoder = new TextDecoder("utf-8");   // created once, outside the loop
let buffer = "";
 
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });   // more bytes to come
  // event-boundary handling goes here (next section)
}
buffer += decoder.decode();   // flush anything still held back

Constructing a fresh TextDecoder per chunk defeats the stream option entirely, because the held-back bytes live in instance state. This is a realistic regression: it appears the moment someone refactors the receive loop into a helper function and moves the constructor inside it.

The second boundary does not corrupt — it deletes

Having fixed the byte layer, I assumed I was finished. The short-ending symptom did not go away.

SSE events are delimited by a blank line. Chunk boundaries are just as indifferent to that delimiter as they are to character boundaries.

I measured two parsers side by side: one that inspects and discards its buffer on every read, and one that accumulates until it sees a blank line.

Write size	Reads	No buffering: events recovered	No buffering: parse failures	Buffered: events recovered
32 bytes	45	0 of 8	5	8 of 8
64 bytes	23	0 of 8	7	8 of 8
128 bytes	12	0 of 8	8	8 of 8
256 bytes	7	3 of 8	5	8 of 8
512 bytes	4	6 of 8	2	8 of 8
1,406 bytes (single write)	2	8 of 8	0	8 of 8

Look at the 512-byte row. Six of eight events arrive. There is no corruption at all. Two events are simply gone.

That was the missing final sentence on my device. Not mangled characters — a whole event dropped.

And the drop is not reported as a failure anywhere. A parser that swallows unparseable fragments — a try followed by continue is entirely ordinary code — does not even increment a counter. The response is 200, the prose reads naturally, and it is short.

Respecting the event boundary is not complicated.

// the body of the while loop above; buffer is held outside it
while (true) {
  const idx = buffer.indexOf("\n\n");
  if (idx === -1) break;                    // delimiter hasn't arrived; wait for the next read
  const block = buffer.slice(0, idx).trim();
  buffer = buffer.slice(idx + 2);           // consume exactly what was handled
  if (!block.startsWith("data:")) continue; // skip comment lines and keepalives
  const payload = block.slice(5).trim();
  if (payload === "[DONE]") continue;
  const obj = JSON.parse(payload);
  const text = obj?.candidates?.[0]?.content?.parts?.[0]?.text;
  if (text) onDelta(text);                  // only now is it safe to render
}

The rule is: never consume what you have not yet delimited. Keep buffer outside the loop, slice off only what indexOf found, and return to reading when it finds nothing.

Why none of this reproduces on your machine

The last row of the measurement table is the whole explanation.

With all 1,406 bytes written at once, even the unbuffered parser recovered all eight events with zero corruption. In local development the response arrives in one piece. No boundaries exist, so boundary bugs cannot fire.

With Python's requests the trap takes a more specific shape.

Call pattern	Result
`iter_lines(chunk_size=16, decode_unicode=True)`	Exact match in every condition
`iter_content(chunk_size=16, decode_unicode=True)`	Exact match in every condition
`iter_content(chunk_size=16)` with a manual `.decode()`	26 U+FFFD
`iter_content(chunk_size=8192)` with a manual `.decode()`	Zero corruption

Given decode_unicode=True, requests runs an incremental decoder internally and you need nothing else. Only the pattern that takes raw bytes and decodes them on the spot breaks.

The last row is the dangerous one. A chunk_size of 8192 shows up in a great many code samples, and it exceeds this entire payload, so exactly one chunk is produced. The incorrect implementation passes with zero corruption.

Production responses run well past 8,192 bytes, at which point boundaries finally exist. That is how you end up with code that is green locally and wrong in the field.

One more detail worth knowing: the 26 replacement characters from iter_content(chunk_size=16) stayed constant whether the server wrote in 16-, 64-, or 256-byte units. requests re-slices the received stream to your requested size, so the boundaries are created independently of any network fragmentation. Read the other way round, that is good news — shrinking chunk_size reproduces the bug deterministically on your laptop.

The adversarial server to keep in CI

Once the reproduction conditions are known, the test writes itself. Reuse server.py and give it an aggressively small write size.

# test_streaming_boundary.py
import subprocess, sys, time, socket
import pytest
 
EXPECTED = (
    "画像分類パイプラインの移行を進めています。停止予定のモデルを洗い出したところ、"
    "参照箇所は思ったより広範囲に散っていました。特に設定ファイル側の既定値が"
    "見落とされがちです。"
)
 
def _free_port():
    with socket.socket() as s:
        s.bind(("127.0.0.1", 0))
        return s.getsockname()[1]
 
@pytest.mark.parametrize("chunk", [8, 16, 32, 64, 128, 512])
def test_streaming_client_survives_fragmentation(chunk):
    """The body must match character for character under heavy fragmentation."""
    port = _free_port()
    srv = subprocess.Popen([sys.executable, "server.py", str(port), str(chunk)])
    time.sleep(0.1)
    try:
        # call the real production client; do not reimplement it here
        from myapp.gemini_stream import collect_stream
        text = collect_stream(f"http://127.0.0.1:{port}/v1beta/stream")
    finally:
        srv.wait(timeout=10)
 
    assert "�" not in text, "replacement characters leaked into the body"
    assert text == EXPECTED, f"body mismatch ({len(text)} chars, expected {len(EXPECTED)})"

The two assertions do different jobs on purpose. Checking only for U+FFFD catches the byte layer and nothing else — a dropped event produces a perfectly clean string that happens to be too short, and sails straight through. Comparing full length and content is what covers both layers.

Use non-ASCII text in the fixture. With an English body, all 40 conditions I measured come back green. The suite passes and the bug ships.

Starting the sweep at 8 bytes is deliberate: that is where detection power is highest, corrupting 24.7% of the body in my runs. Real networks rarely fragment that finely, but a test is the cheapest place there is to reproduce a rare condition.

Checking for the final event is not enough to notice a loss

Once I knew events could vanish, my first idea for a watchdog was the obvious one. In Gemini's stream, only the last event carries finishReason and usageMetadata. If those never arrive, something was lost.

It takes a few lines and it matches intuition. But when I re-measured the same conditions against the mock server, that check missed a case.

Write size	Events received (non-buffering client)	Parse failures swallowed	`finishReason`	Caught by terminal check	Caught by failure counter
32 bytes	0 / 8	5	missing	caught	caught
64 bytes	0 / 8	7	missing	caught	caught
128 bytes	0 / 8	8	missing	caught	caught
256 bytes	3 / 8	5	missing	caught	caught
512 bytes	6 / 8	2	arrived	missed	caught
1,406 bytes (one write)	8 / 8	0	arrived	clean	clean

The 512-byte row is the point of this section. The final event arrived intact, so finishReason is visible. Two events in the middle are still gone, and only 66 of the 85 characters of body text survived.

A terminal check tells you whether the stream reached its end. It does not tell you whether everything in between is present. Those are different questions. A connection that drops mid-stream trips the terminal check; a buffering mistake does not.

What worked was counting the failures I had been swallowing. Add one counter to the catch of the try / continue pattern.

let swallowed = 0;   // data blocks that failed to parse and were dropped
 
// ... after cutting a block at the event delimiter
let obj;
try {
  obj = JSON.parse(payload);
} catch (e) {
  swallowed += 1;    // count it before dropping it
  continue;
}
 
// when the stream ends
if (swallowed > 0) {
  reportMetric("sse_parse_swallowed", swallowed);   // use whatever telemetry you already have
}

A receive loop that respects the event boundary only cuts a block after its delimiter has arrived, so an unparseable block should never occur. Any value of swallowed above zero therefore points at a boundary mistake somewhere. In the table above, the counter was non-zero in all five lossy conditions and stayed at zero for the single-write run.

One caveat: that correspondence holds for this mock server's conditions. On a real route, a proxy may cut a response short, or a server may genuinely return malformed JSON, and the counter will rise then too. Telling your own implementation apart from the route is what the eight-byte mock server is for.

I had assumed a terminal check would be enough and that a counter was overkill. It was only after running the 512-byte case that I saw the gap. Count a failure before you swallow it. Whenever I now see a catch whose whole body is continue, that is where the counter goes.

A diagnosis order that saves time

If any of this sounds familiar, this is the sequence I would work through.

Compare lengths. Send the same prompt without streaming and diff the character counts. Short output points at the event boundary; embedded U+FFFD points at the byte boundary. Both can be true at once
Find where the decoder is constructed. A TextDecoder or incremental decoder created inside the read loop cannot hold state across chunks. One instance per request
Check how the buffer is drained. If it is cleared on every read, events will be lost. It should be sliced only up to a delimiter that has actually arrived
Shrink the chunk size and run it locally. Sixteen bytes with requests, or eight bytes with the mock server. If it does not reproduce there, the cause lives somewhere else
Look for a catch that only continues. Add a counter there and watch whether it ever rises above zero in production for a week

I got through step two, saw the corrupted characters disappear, and very nearly closed the investigation. What was actually left was step three. The failure mode had changed from corruption to deletion, which looked a great deal like a fix.

A change in how something breaks is not the same as it being fixed, and streaming makes that distinction easy to miss.

The value of streaming is that you can show partial results. Once you commit to rendering an in-flight response, boundary handling stops being plumbing and becomes part of your output quality. Having the numbers on hand turns a puzzling device-only symptom into a fifteen-minute fix.

Thank you for reading — I hope the harness saves you some time.

Thank You for Reading

Gemini Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

✦Copy-paste ready implementation code
✦New advanced guides published daily
✦$5/mo or $10 for lifetime access

View Membership →