Should Omni Flash Hand You the 4K, or Should You Upscale at the Last Step?

Checking whether the detail matches the nominal size

If the dimensions are identical, look at the frequency domain instead. Footage stretched up from 720p carries almost no energy above the 720p Nyquist frequency. Measure that band and you can tell whether the nominal size is backed by real detail.

"""Judge whether detail matches the nominal resolution, relative to a 720p reference.
Usage: python3 native_check.py candidate.mp4 reference_720p.mp4
"""
import subprocess
import sys
 
import numpy as np
 
 
def grab_frame(path, at="00:00:01"):
    """Pull one frame at the given timestamp as raw grayscale."""
    wh = subprocess.run(
        ["ffprobe", "-v", "error", "-select_streams", "v:0",
         "-show_entries", "stream=width,height", "-of", "csv=p=0", path],
        capture_output=True, text=True, check=True).stdout.strip().split(",")
    w, h = int(wh[0]), int(wh[1])
    raw = subprocess.run(
        ["ffmpeg", "-v", "error", "-ss", at, "-i", path, "-frames:v", "1",
         "-pix_fmt", "gray", "-f", "rawvideo", "-"],
        capture_output=True, check=True).stdout
    return np.frombuffer(raw, np.uint8).reshape(h, w).astype(np.float64), w, h
 
 
def detail_density(frame, w, h):
    """High-frequency energy per pixel. Divide by area rather than by total
    energy so clips of different sizes stay comparable."""
    win = np.hanning(h)[:, None] * np.hanning(w)[None, :]  # kill edge discontinuity
    spec = np.abs(np.fft.fftshift(np.fft.fft2(frame * win))) ** 2
    fy = (np.arange(h) - h // 2) / h
    fx = (np.arange(w) - w // 2) / w
    r = np.sqrt(fy[:, None] ** 2 + fx[None, :] ** 2)
    band = spec[(r > 0.25) & (r <= 0.5)].sum()   # the band stretching leaves empty
    return float(band / (w * h))
 
 
if __name__ == "__main__":
    cf, cw, ch = grab_frame(sys.argv[1])
    rf, rw, rh = grab_frame(sys.argv[2])
    cd = detail_density(cf, cw, ch)
    rd = detail_density(rf, rw, rh)
    gain = cd / rd if rd > 0 else float("inf")
    label = "upscaled" if gain < 3 else "detail matches size"
    print(f"{sys.argv[1]}: nominal {cw}x{ch} / detail ratio {gain:.2f}x -> {label}")

Run against the three clips, using a 720p render of the same picture as the reference:

Nearly a 97x spread between the detailed native clip and the upscaled one. That is a gap you can act on without squinting at frames.

Fixing one thing to compare against is the same shape I ended up with when classifying wallpapers. I wrote up the parallel-model version of that in running Gemini 3 Pro and 2.5 Pro side by side on wallpaper classification.

The absolute threshold did not work

My first version had no reference clip. It computed the share of total energy sitting above the cutoff and called anything under 0.01 an upscale. The results:

All three came back as upscaled, and the third one — a genuine native 4K render — pinned at zero. A flat picture has nothing in the high band whether it was authored large or not.

That pushed me to change the question. What this measurement answers is not "was it authored natively" but "does this file carry detail worth its size" — and in practice, the second question is the one I actually need answered. So the reference line comes from outside now. Generate one extra 720p clip and every later comparison can reuse it.

I have a habit of reaching for absolute thresholds, because a single number in a config file feels tidy. But putting a threshold on a metric that moves with the material builds something that misjudges quietly, forever. I wrote about a similar observability trap in why the AI Studio developer log should not be your source of truth.

My assumption about double upscaling was wrong

The other thing I wanted to check was whether splitting the upscale across two steps degrades the picture. When the delivery target is 1080p, there are two routes:

1. Ask the API for 4k, then shrink to 1080p yourself — an enlargement plus a reduction
2. Take 720p as-is and enlarge once, right before delivery

I expected route 1 to come out softer. Building both and comparing them, route 2 measured 1.02x the detail density of route 1, and the PSNR between the two frames was 40.79 dB. That is not a difference anyone will see.

So the choice of upscale point is not a picture-quality decision. I want to be straight about that, because my prior was wrong. What actually differs is payload and wait time.

Same material, same CRF: the 720p clip is 402,983 B, the nominal 4K clip is 1,812,869 B. 4.50x. The Interactions API returns generated video as base64, so that difference lands directly in your response body — roughly 2.3 MiB versus 0.5 MiB. That is for two seconds of footage. Multiply by the number of takes you burn through and it stops being a rounding error.

So what does decide it

If picture quality is off the table, three things are left:

• The size your delivery target accepts. When a store's preview video spec makes dimensions a condition of acceptance, there is nothing to decide
• Payload. While iterating on takes, receiving 720p and normalizing once downstream is simply lighter
• Whether a post-processing step already exists. If you are running ffmpeg anyway, that is where the upscale belongs

Writing this out as configuration keeps the answer stable across runs:

NATIVE_CEILING_H = 720          # the highest the API renders natively
 
def choose_resolution(target_h: int, has_postprocess: bool) -> tuple[str, str]:
    """Return (resolution to request, where to upscale).
    target_h is the vertical pixel count the delivery target requires.
    """
    if target_h <= 360:
        return "360p", "none"          # a downscale alone will do
    if target_h <= NATIVE_CEILING_H:
        return "720p", "none"          # native territory
    if has_postprocess:
        # ffmpeg runs regardless, so take the lighter payload
        return "720p", "local"
    # no post-processing stage: let the API match the required size
    return ("1080p" if target_h <= 1080 else "4k"), "api"
 
 
for target_h, post in [(720, False), (1080, True), (1080, False), (2160, False)]:
    res, point = choose_resolution(target_h, post)
    print(f"target={target_h}p postprocess={post} -> resolution={res} upscale_at={point}")

Which prints:

target=720p postprocess=False -> resolution=720p upscale_at=none
target=1080p postprocess=True -> resolution=720p upscale_at=local
target=1080p postprocess=False -> resolution=1080p upscale_at=api
target=2160p postprocess=False -> resolution=4k upscale_at=api

The point is that upscale_at always resolves to exactly one place. Leave it ambiguous across several scripts and you end up with a path where the API stretches to 4K and a later stage stretches again. The quality loss turned out to be mild, as measured — but transfer volume and generation wait stack up honestly.

Required dimensions on the App Store and Google Play do change. Keep those numbers out of your code and in per-target configuration, so a store spec change means editing one place. I collected the device-resolution handling from my wallpaper work in generating wallpaper variations with Gemini 3.2 Flash Image Output.

Fix the resolution before you start extending

Omni Flash can also extend a clip. Prompt it to continue the scene and it generates a 3 to 10 second continuation. Videos it generated earlier are referenced with previous_interaction_id; your own footage goes in through the Files API.

If extension is part of the plan, settle the resolution when you cut the first shot. Changing resolution midway adds a normalization step later, and because one segment was shrunk to match while another was not, the texture can visibly shift at the seam.

The sequence I use:

1. Iterate on framing and timing at 360p, where cost and latency are lowest
2. Once the structure holds, move to 720p and run extensions at the same setting
3. Only after every shot exists, upscale once to each delivery target

Extension constraints are still being updated in the docs. Reading them before you build saves a round of rework.

delivery: "uri" still returns base64 on GET

There is one more behavior worth knowing on the retrieval side. Even when an interaction was created with delivery: "uri", GET /v1beta/interactions/{id} returns the video as inline base64 in the data field. The uri field is only guaranteed on the initial creation response or in the SSE stream.

If your design generates asynchronously and fetches by ID later, that is where an unexpected payload arrives — and if you asked for 4K, a heavy one.

The fix is simple: if you need the URI, capture it at creation time. Do not write code that assumes you can go back for it.

interaction = client.interactions.create(
    model="gemini-omni-1.1-flash",
    input="A drone shot of a mountain landscape at sunrise.",
    response_format={"type": "video", "resolution": "720p"},
)
 
# grab the uri now — a later GET hands back base64 instead
video = interaction.output_video
uri = getattr(video, "uri", None)
if uri:
    save_uri_to_manifest(interaction.id, uri)
else:
    save_bytes(interaction.id, base64.b64decode(video.data))

Put the migration deadline on the calendar

gemini-omni-flash-preview shuts down on September 30, 2026. It entered public preview on June 30, so its whole life is about three months. If production code still points at the preview endpoint, you have roughly a month.

The swap itself is usually just a model name. But now that the default resolution is documented as 720p, anything that leaned on undocumented preview behavior may come out different. After swapping, run ffprobe once and confirm the dimensions. Shutdown dates are listed on the Gemini API deprecations page.

I have already had code stop working quietly when an image generation model was retired. Doing the work while the date is still published turned out to be the cheaper option.

Where to start

Generate one clip at 720p and keep it as your reference. Both the detail check and the upscale-point decision get easier the moment that file exists.

Choosing a resolution is not an operation that improves the picture. It is an operation that matches dimensions to what the receiving end will accept. Keep that order and both the cases where 4k is right and the cases where it is not become easy calls.

I have been doing exactly this with wallpaper delivery for years, and a new API still had me reasoning from the source side first. Thanks for reading.