Gemma 4 × OpenCode Advanced Guide

Getting Gemma 4 × OpenCode "working" is the easy part. Getting it to a point where it's actually useful in a real development workflow takes more deliberate setup.

Context window limits, prompt design for smaller models, and the judgment of which tasks belong on local versus cloud LLMs — these aren't things you figure out automatically. Without working through them, it's easy to conclude "local LLMs aren't worth it" before you've given them a fair test.

Choosing the Right Gemma 4 Variant

Four model sizes exist. Picking the right one matters.

E2B (up to ~4GB RAM)

The smallest variant. Designed for mobile and IoT integration. Not practical for coding tasks — limited instruction-following and code quality.

E4B (up to ~8GB RAM)

The mid-range edge model. Runs comfortably on Apple Silicon M1+ Macs and Windows machines with 16GB RAM.

Small-to-medium function implementations, code review comments, and documentation generation all fall within usable accuracy. Works well as a "sub-engine" for development tasks where API cost is the constraint.

26B MoE (up to ~20GB unified/VRAM)

Uses Mixture-of-Experts architecture — surprisingly high accuracy relative to its parameter count. Requires Apple Silicon M2 Ultra or higher, or a GPU with 24GB+ VRAM.

Multi-file refactoring and architecture review tasks become more viable here.

31B Dense (~35GB RAM)

Highest accuracy. Not realistic for personal machines. Intended for server deployment.

Recommended path for individuals: Start with E4B, validate the speed-quality balance for your use case, then step up to 26B MoE if needed.

Practical Context Window Management

The most common failure point with local LLM coding agents is running out of context. Ollama's default 4,096-token window fills up quickly with real codebases.

# For E4B (16GB RAM)
cat > Modelfile-e4b << 'EOF'
FROM gemma4:e4b
PARAMETER num_ctx 32768
PARAMETER num_gpu 99
EOF
ollama create gemma4-e4b-coder -f Modelfile-e4b
 
# For 26B MoE (32GB+ RAM)
cat > Modelfile-26b << 'EOF'
FROM gemma4:26b
PARAMETER num_ctx 65536
PARAMETER num_gpu 99
EOF
ollama create gemma4-26b-coder -f Modelfile-26b

num_gpu 99 pushes as many layers as possible to GPU memory. On Apple Silicon, this means nearly all layers stay on the neural engine.

Directory Strategy to Conserve Context

For large projects, starting OpenCode from a specific subdirectory — rather than the project root — keeps context focused and effective.

# Start in the module you're actually working on
cd src/modules/payment
opencode
 
# Or specify what to exclude from context
cat > .opencodeignore << 'EOF'
node_modules/
.next/
dist/
*.test.ts
*.spec.ts
*.log
EOF

Excluding test files and build artifacts alone frees up substantial context for actual source code.

Prompt Design for Local LLMs

Prompts that work well with Claude or Gemini 1.5 Pro often don't translate cleanly to local models. Smaller models respond better to different patterns.

Principle 1: Make tasks as atomic as possible

❌ "Refactor this entire module to improve type safety and add comprehensive tests."

✅ "Change the return type of the fetchUser function from User | null 
   to Result<User, ApiError>. Modify only this file."

Local models lose coherence on large, multi-goal tasks. One clear task per prompt produces substantially more reliable output.

Principle 2: Specify the output format explicitly

Add JSDoc comments to the following function.
Output format: only the complete code with JSDoc added. No explanatory text.

[code]

"No explanatory text" suppresses the tendency smaller models have to pad responses with verbose introductions.

Principle 3: Reference specific files

Refer to @src/types/user.ts and @src/api/fetchUser.ts.
Fix the type error in the fetchUser return value.

"Look at the whole codebase" produces worse results than explicit file references. The model processes what you point at, rather than trying to figure out what's relevant.

Hybrid Cloud-Local Workflow

In practice, the most balanced approach is distributing tasks between local and cloud models based on complexity and sensitivity.

Give to Local (Gemma 4)

Comment and documentation generation
Small function implementations (under 50 lines)
Variable and function rename suggestions
Simple bug fixes where the error message is clear
Boilerplate and scaffolding generation

Give to Cloud (Claude / Gemini)

Implementations requiring design judgment
Multi-file refactoring
Security-sensitive code review
Complex algorithm implementation
Any code that can safely leave your machine

Writing this decision framework into a CODING_POLICY.md at your project root keeps the logic consistent — and communicable if you work with others.

Managing Multiple Profiles in OpenCode

// ~/.config/opencode/config.json
{
  "profiles": {
    "local-fast": {
      "provider": "ollama",
      "model": "gemma4-e4b-coder",
      "baseUrl": "http://localhost:11434"
    },
    "local-quality": {
      "provider": "ollama",
      "model": "gemma4-26b-coder",
      "baseUrl": "http://localhost:11434"
    },
    "cloud": {
      "provider": "anthropic",
      "model": "claude-sonnet-4-6"
    }
  },
  "defaultProfile": "local-fast"
}

Switch profiles with:

opencode --profile cloud

The Right Mental Model for Local LLMs

The key to using Gemma 4 × OpenCode productively is not treating it as a drop-in Claude Code replacement. It's a zero-cost sub-engine that handles specific, well-defined tasks — while your cloud model handles the rest.

With model selection, context configuration, and prompt design aligned, the local environment performs well above the "toy project" level. Start with E4B, find where it fits in your workflow, and expand from there.

Gemma 4 × OpenCode Advanced Guide: Building a Production-Ready Local AI Dev Environment