Two Kinds of Video Questions: Why I Send "Find It" and "Prove It Isn't There" Down Separate Paths

Sorting questions into two piles

The top two resolve the moment something is found. They never need to claim "there is none," so segments the model skipped cannot corrupt the answer.

The bottom two are the opposite. The instant "we watched all of it" stops being true, the answer stops meaning anything. The same three words — no matches found — are a useful reply above the line and an unverified claim below it.

Find-it questions go straight through

For searching, I use the official shape as-is. The only thing I add is writing the usage figures down next to the answer.

import json
import time
from pathlib import Path
 
from google import genai
 
client = genai.Client()
LOG = Path("video_probe_log.jsonl")
 
 
def ask_agentic(video_uri: str, question: str, model: str = "gemini-3.7-flash") -> str:
    started = time.time()
    interaction = client.interactions.create(
        model=model,
        input=[
            {"type": "video", "uri": video_uri, "processing": "agentic"},
            {"type": "text", "text": question},
        ],
    )
    # attribute naming shifts between SDK releases, so try both before giving up
    usage = getattr(interaction, "usage", None) or getattr(interaction, "usage_metadata", None)
    record = {
        "ts": time.strftime("%Y-%m-%dT%H:%M:%S"),
        "model": model,
        "mode": "agentic",
        "video": video_uri,
        "question": question,
        "elapsed_sec": round(time.time() - started, 2),
        "usage": usage.to_json_dict() if hasattr(usage, "to_json_dict") else str(usage),
        "answer": interaction.output_text,
    }
    with LOG.open("a", encoding="utf-8") as f:
        f.write(json.dumps(record, ensure_ascii=False) + "\n")
    return interaction.output_text

There is a reason the usage goes in every time. Under agentic processing, the same video and the same model consume different amounts depending on how the question is shaped. When you later need to rebuild a cost estimate, your own logged history is the only thing you can lean on.

Reaching for the attribute under two different names is the kind of insurance preview features tend to need. If one returns None I would still rather keep the record, even as a string, than lose the row.

Prove-absence questions get cut up on my machine first

You can name a time range through API parameters, but I cut the file locally instead. The clips stay on my disk, which means "this stretch was definitely reviewed" exists as an artifact I can hand back. On client work especially, being able to retrace a check months later matters more than saving a step.

I found the same thing with audio: doing a pass with my own tools before handing anything to Gemini made the decision faster, which I wrote about in catching loop-point clicks at the waveform edge before asking Gemini to listen.

The windows come first, and the coverage check comes before anything is uploaded.

def plan_windows(duration_sec: float, window_sec: float = 300.0,
                 overlap_sec: float = 2.0) -> list[tuple[float, float]]:
    """Return (start, end) pairs covering [0, duration].
    The overlap keeps events that straddle a boundary from being halved."""
    if duration_sec <= 0:
        raise ValueError("duration_sec must be positive")
    if overlap_sec >= window_sec:
        raise ValueError("overlap_sec must be smaller than window_sec")
 
    step = window_sec - overlap_sec
    windows: list[tuple[float, float]] = []
    start = 0.0
    while start < duration_sec:
        end = min(start + window_sec, duration_sec)
        windows.append((round(start, 3), round(end, 3)))
        if end >= duration_sec:
            break
        start += step
    return windows
 
 
def assert_covers(windows, duration_sec: float, tolerance: float = 1e-6) -> None:
    """Raise unless the windows cover [0, duration] without gaps."""
    if not windows:
        raise AssertionError("no windows")
    ordered = sorted(windows)
    if ordered[0][0] > tolerance:
        raise AssertionError(f"head gap: 0.0 .. {ordered[0][0]}")
 
    reached = ordered[0][1]
    for start, end in ordered[1:]:
        if start > reached + tolerance:
            raise AssertionError(f"gap: {reached} .. {start}")
        reached = max(reached, end)
 
    if reached < duration_sec - tolerance:
        raise AssertionError(f"tail gap: {reached} .. {duration_sec}")
 
 
# A 63-minute recording, five-minute windows, two seconds of overlap
windows = plan_windows(3_780.0, window_sec=300.0, overlap_sec=2.0)
assert_covers(windows, 3_780.0)
print(len(windows), windows[0], windows[-1])
# -> 13 (0.0, 300.0) (3576.0, 3780.0)

assert_covers runs first so that changing the window math surfaces a hole immediately. Shrink window_sec or nudge overlap_sec and you can open a gap without noticing — and a sweep with a hole in it still runs to completion and still prints an answer. This is the spot where I want a machine to stop me.

Cutting is left to ffmpeg.

import subprocess
 
 
def cut_windows(src: str, windows, out_dir: str = "windows") -> list[str]:
    Path(out_dir).mkdir(exist_ok=True)
    paths = []
    for i, (start, end) in enumerate(windows):
        out = f"{out_dir}/win_{i:03d}.mp4"
        subprocess.run(
            [
                "ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
                "-ss", f"{start}", "-to", f"{end}", "-i", src,
                "-c", "copy", out,
            ],
            check=True,
        )
        paths.append(out)
    return paths

Sweeping, with coverage reported alongside the verdict

Each window goes through static processing. Leaving processing out returns you to the default, so the sweep is actually one line shorter than the search.

def sweep_absence(window_files, question, done_path="sweep_done.jsonl"):
    """Scan every window with static processing.
    A single missing window yields 'undetermined' rather than 'none found'."""
    p = Path(done_path)
    done: dict[str, str] = {}
    if p.exists():
        for line in p.read_text(encoding="utf-8").splitlines():
            rec = json.loads(line)
            done[rec["window"]] = rec["answer"]
 
    for path in window_files:
        if path in done:
            continue
        interaction = client.interactions.create(
            model="gemini-3.7-flash",
            input=[
                {"type": "video", "uri": upload_file(path)},  # no processing key = static
                {"type": "text", "text": question},
            ],
        )
        done[path] = interaction.output_text
        with p.open("a", encoding="utf-8") as f:
            f.write(json.dumps({"window": path, "answer": done[path]},
                               ensure_ascii=False) + "\n")
 
    missing = [w for w in window_files if w not in done]
    hits = [(w, a) for w, a in done.items() if not a.startswith("NONE")]
 
    if missing:
        return {"verdict": "undetermined", "covered": len(done),
                "total": len(window_files), "missing": missing, "hits": hits}
    return {"verdict": "found" if hits else "none",
            "covered": len(done), "total": len(window_files), "hits": hits}

The three-way return value is the point. Putting undetermined between found and none means an interrupted sweep can never be misread as a clean result. I print covered / total in the output too — I wanted twelve-of-thirteen to look like twelve of thirteen.

If transcripts feed into the judgment, it is worth checking how your transcription behaves before you rely on it. Some configurations quietly withhold speaker labels and word-level timestamps, which I documented in why smart mode in gemini-3.5-transcribe returns neither diarization nor word timestamps.

Searching is something I can delegate. Deciding that the search is finished is still my job.

Your cost estimate stops being a function of duration

Under static processing, tokens fell out of duration, FPS, and resolution. You could work out the monthly figure before uploading anything.

Agentic processing breaks that arithmetic. How much the model traverses depends on the question, so the same hour of footage costs differently depending on how easy it is to find what you asked for.

So I name my questions as templates and accumulate real usage per template. What I estimate from is not the mean but the median and the upper tail — the tail is what decides the invoice, and an estimate that ignores it will be wrong.

It also helps to remember what the up-to-88% figure is measured against: static sampling at one frame per second. On questions with thin handholds the model ranges more widely and the gap narrows. Since switching costs nothing extra, the risk is not the switch — it is continuing to use the old estimating formula afterwards.

Where I currently draw the line

That last row has turned out to be the most useful one. Running both costs no extra feature fee, and the disagreements are a map of where the model chose not to look. As an indie developer splitting time between my own apps and client work, I have a fixed and small budget for reviewing footage — a map that narrows the rewatch is worth a lot.

Things worth knowing before you build this

• Putting -ss before -i with -c copy snaps the start to a keyframe boundary. When the check needs second-level precision, I re-encode with -c:v libx264 instead
• With zero overlap, an event straddling a boundary shows up as two halves and gets missed by both. I start at two seconds and lengthen it for fast-moving footage
• Matching on a literal phrase to detect "nothing found" is brittle. Pin the response to a JSON shape and compare fields — the code above favours readability, and in production I use structured output
• Very small windows drive up call counts until static processing gets expensive too. I start around five minutes and adjust to the material

What to do next

Pick one long video you already have, write out the questions you habitually ask it, and sort them into find-it and prove-absence. By the time the sorting is done, which processing mode each one belongs to has mostly answered itself.

Thank you for reading. Coverage was not a word I thought about until this particular hour of footage taught me to, and if it saves you one version of that afternoon, I will be glad.