Building Automatic Wallpaper Category Classification with Gemini Vision

While preparing the Android v2.0.0 release of Beautiful HD Wallpapers, I found myself struggling with image categorization. The app contains thousands of wallpapers spanning 30+ categories — "nature," "city," "abstract," "animals," "space," and more — and getting them accurately sorted had been a persistent challenge. I decided to test how far Gemini Vision API could take me.

Short answer: the accuracy exceeded my expectations, but the model's behavior with ambiguous images wasn't something the official docs prepared me for. Here's what I learned.

Why Gemini Vision

I had a few options for automated classification:

Local models like MobileNet or EfficientNet are fast and cheap, but their category definitions are fixed and struggle with nuanced labels like "mystical forest" or "watercolor-style cityscape." GPT-4o Vision delivers solid accuracy but costs roughly 2–3x more per request than Gemini 2.0 Flash.

Gemini 2.0 Flash won on the cost-to-quality tradeoff. When you're processing thousands of images as an indie developer, that math matters. As of May 2026, image + text input with Gemini 2.0 Flash runs around $0.0001 per 1,000 tokens — significantly cheaper than GPT-4o Vision.

Basic Implementation

Here's a minimal working implementation for wallpaper classification:

import google.generativeai as genai
from PIL import Image
import json
 
genai.configure(api_key="YOUR_GEMINI_API_KEY")
 
CATEGORIES = [
    "nature", "city", "abstract", "animals", "space",
    "ocean", "mountain", "forest", "flower", "architecture",
    "art", "minimal", "dark", "colorful", "japan",
]
 
def classify_wallpaper(image_path: str) -> dict:
    """
    Classify a wallpaper image using Gemini Vision.
    Returns: {"category": "nature", "confidence": "high", "reason": "..."}
    """
    model = genai.GenerativeModel("gemini-2.0-flash")
    image = Image.open(image_path)
 
    prompt = f"""
You are a wallpaper app category classifier.
Select exactly one category for this image from the following list:
{', '.join(CATEGORIES)}
 
Respond in this JSON format only:
{{
  "category": "selected category",
  "confidence": "high / medium / low",
  "reason": "one-sentence explanation"
}}
 
Return JSON only — no additional explanation.
"""
 
    response = model.generate_content([prompt, image])
 
    text = response.text.strip()
    if "```json" in text:
        text = text.split("```json")[1].split("```")[0].strip()
    elif "```" in text:
        text = text.split("```")[1].split("```")[0].strip()
 
    return json.loads(text)
 
# Example usage
result = classify_wallpaper("wallpaper_001.jpg")
print(result)
# Output: {"category": "nature", "confidence": "high", "reason": "Morning mist over a dense green forest"}

This works — but in production, you'll hit issues quickly.

Real Problems and Solutions

JSON Parse Failures

Gemini 2.0 Flash sometimes returns text outside the JSON structure, even when instructed not to. When confidence is low, the model occasionally prepends something like "This image is difficult to classify. The closest category would be..." before the JSON.

A naive json.loads() call failed on 10–15% of responses in production. Adding a regex-based extractor solved it:

import re
 
def extract_json_safely(text: str) -> dict | None:
    """Safely extract JSON from a response that may contain surrounding text."""
    text = re.sub(r'```(?:json)?\n?', '', text).strip()
 
    json_match = re.search(r'\{[^{}]*\}', text, re.DOTALL)
    if not json_match:
        return None
 
    try:
        return json.loads(json_match.group())
    except json.JSONDecodeError:
        return None

Ambiguous Category Boundaries

Images like "abstract watercolor forest" or "dark underwater that looks like space" were consistently unstable across runs. Adding explicit exclusion examples for each category to the prompt made a significant difference:

CATEGORY_DESCRIPTIONS = {
    "nature": "Forests, meadows, rural landscapes. If buildings are visible, use architecture or city instead.",
    "ocean": "Sea, coastline, waves. Aquarium underwater photos count. Rivers and lakes go under nature.",
    "abstract": "Patterns, gradients, digital art where no real-world objects are recognizable.",
    "art": "Paintings, illustrations, anime-style art, heavily processed photos that look like paintings.",
}

After adding these descriptions, medium-confidence accuracy improved from 78.6% to 87.3%.

Rate Limiting at Scale

Sending thousands of requests in quick succession triggers Gemini API rate limits (free tier: 15 RPM / 1,000 RPD). Even on paid plans, aggressive parallelism causes throttling.

import time
 
def classify_batch(image_paths: list, rpm_limit: int = 30) -> list:
    """
    Process images in batch while respecting rate limits.
    rpm_limit: max requests per minute
    """
    results = []
    delay = 60 / rpm_limit
 
    for i, path in enumerate(image_paths):
        result = classify_wallpaper(path)
        results.append({"path": path, **result})
 
        if (i + 1) % 10 == 0:
            print(f"Progress: {i + 1}/{len(image_paths)}")
 
        if i < len(image_paths) - 1:
            time.sleep(delay)
 
    return results

Accuracy Results on Real Data

I tested against 2,400 actual wallpapers from Beautiful HD Wallpapers:

• High-confidence classifications: 94.2% accuracy
• Medium-confidence classifications: 78.6% → 87.3% (after category description improvements)
• Low-confidence classifications: 51.3% accuracy

By routing confidence: low images to a manual review queue, I achieved an overall automation rate of ~92%. Before Gemini Vision, manual classification topped out at 200–300 images per day. With batch processing, I can now handle 1,000+ images per hour.

Cost Comparison: Gemini 2.0 Flash vs GPT-4o Vision

Running the same 2,400 images through both models:

• Gemini 2.0 Flash: ~$0.18 total
• GPT-4o Vision: ~$0.54 total (same conditions)

For an indie developer, a 3x cost difference isn't trivial. The accuracy gap at high-confidence was only 1–2%. For wallpaper classification specifically, Gemini 2.0 Flash is more than sufficient. If you need absolute precision on nuanced categories and budget isn't a constraint, GPT-4o Vision is slightly more stable — but the difference is marginal.

Cutting JSON Parse Failures at the Source with Structured Output

The implementation above rescues JSON with regex. Since then, SDK-side structured output has matured to the point where parse failures barely happen at all. With the newer google-genai SDK, passing a Pydantic model to response_schema makes the model return JSON that conforms to that schema and nothing else.

from google import genai
from pydantic import BaseModel
from enum import Enum
 
class Confidence(str, Enum):
    high = "high"
    medium = "medium"
    low = "low"
 
class Classification(BaseModel):
    category: str
    confidence: Confidence
    reason: str
 
client = genai.Client(api_key="YOUR_GEMINI_API_KEY")
 
def classify_structured(image_path: str) -> Classification:
    img = client.files.upload(file=image_path)
    resp = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=["Classify this wallpaper into a single category.", img],
        config={
            "response_mime_type": "application/json",
            "response_schema": Classification,
        },
    )
    return Classification.model_validate_json(resp.text)

After switching to schema-based output, json.loads() failures caused by prepended text dropped to essentially zero in my setup. Constraining confidence with an Enum also prevents stray values like "very high". Keep the regex fallback from earlier only as insurance for older models that don't support response_schema. Pick the model name that fits your environment — classify in bulk with a lightweight Flash tier, then re-classify only the contested images with a higher tier to balance cost and accuracy.

Where to Draw the Line Between Automation and Human Judgment

After running a wallpaper app for years, what stays with me is that "beauty" holds something you can't fully quantify. Among the images Gemini returns as medium or low, there are genuinely ones the eye is drawn to — and some high ones that look flat once you line them up. Correct classification and actual appeal are simply different things.

That's why I keep my production setup at 92% automated, 8% human. Images that can be judged clearly go straight to the model; human attention is reserved for the confidence: low review queue. For a one-person operation, that line is the most sustainable.

One practical habit paid off more than anything: logging the reason field. When a misclassification happens, being able to re-read why the model chose a category tells me exactly which line of the prompt to fix. Instead of chasing a big accuracy jump, I tune category definitions one sentence at a time, guided by reason. That turned out to be the fastest route, even though it looks like the slowest.

Where to Start

Get an API key from Google AI Studio and run the code above. The free tier lets you process hundreds of images before hitting limits — more than enough to validate the approach for your use case.