With Gemma 4 released in April 2026, local LLM coding finally crossed the line into "actually usable." The 26B A4B variant — 26B total parameters with 4B active — strikes a great balance between latency and quality and runs comfortably on Apple Silicon Macs. Pair it with the open-source coding agent OpenCode and you get an experience close to "free Claude Code."
I use Claude Code in production every day, but I have been quietly testing whether expensive tasks could be offloaded to a local model. This post walks through getting Gemma 4 26B A4B + OpenCode running on Mac or Linux, including the early gotchas that cost me time.
Hardware first: check the requirements
Gemma 4 26B A4B is a mixture-of-experts model with only 4B active parameters per token, which makes it dramatically lighter than a 26B dense model. It still needs real memory to run smoothly.
The minimum I'd recommend (Q4 quantization): 24 GB of unified memory on Apple Silicon, or 16 GB of VRAM on NVIDIA hardware. On my M3 Max (48 GB), Q5_K_M lands around 30 tokens/sec. On an RTX 4090 (24 GB VRAM), expect 60+ tokens/sec at Q4_K_M.
You can technically run it on a 16 GB Mac, but you'll need Q3 quantization and the code-quality drop becomes obvious. I wouldn't recommend it.
Step 1: Pick an inference backend
You have several options. My rule of thumb: Ollama if you're starting out, llama.cpp or LM Studio for steady single-user runs, vLLM if you're hosting for multiple users.
Ollama: Simplest setup. brew install ollama on Mac, one-line installer on Linux. Less tunable, but talks to OpenCode without friction.
LM Studio: GUI for switching quantization variants. Useful if you want to compare quality across builds.
vLLM: The right call when you're serving multiple developers from one inference host.
For this guide we'll use Ollama.
# Mac
brew install ollama
# Linux
curl -fsSL https://ollama.com/install.sh | sh
# Pull Gemma 4 26B A4B
ollama pull gemma4:26b-a4bThe pull is roughly 15 GB at Q4_K_M, so expect 10–20 minutes the first time. When it finishes, ollama run gemma4:26b-a4b should drop you into an interactive shell.
Step 2: Install and configure OpenCode
OpenCode is a coding-agent CLI distributed via npm.
npm install -g @opencode/cli
opencode initopencode init creates ~/.config/opencode/config.json. Edit it to point at Gemma 4 through Ollama:
{
"providers": {
"local-gemma4": {
"type": "ollama",
"host": "http://localhost:11434",
"model": "gemma4:26b-a4b"
}
},
"defaultProvider": "local-gemma4",
"agents": {
"planner": {
"provider": "local-gemma4",
"systemPrompt": "You are a coding planner. Break user requests into implementable steps."
},
"builder": {
"provider": "local-gemma4",
"systemPrompt": "You are a coding implementer. Follow the plan and propose minimal diffs."
}
}
}OpenCode supports multi-agent setups — planner / builder, in this case — which feels a lot like Claude Code's subagent model.
Step 3: Smoke test with a small refactor
Open a real repository and start OpenCode:
cd ~/projects/my-repo
opencodeOnce you're in the interactive prompt, try something like:
"Read src/utils/date.ts and add a helper for ISO 8601 strings."
Gemma 4 26B A4B will read the file and propose a diff. The first thing you'll notice compared to Claude Code is response variance — anywhere from 2 to 10 seconds — but the actual output quality is solid.
If nothing happens, the most common culprits are an Ollama host mismatch or a typo in the model name. Run ollama list and make sure the name in config.json matches exactly.
Step 4: The three things that tripped me up first
Setup is straightforward, but three issues will likely bite you within the first week of real use:
KV cache OOMs. Pass a long context (over 4K tokens) and Ollama can OOM. Set OLLAMA_NUM_CTX=8192 (or higher) explicitly to size the KV cache properly.
File-write friction. OpenCode prompts before every file edit by default. For a trusted local environment, add auto_approve: ["edit", "write"] to config.json to get something close to Claude Code's auto mode.
Prompt language stability. Gemma 4 is optimized for English. Long Japanese instructions sometimes go off the rails. I keep system prompts in English and write user-facing prompts in whatever language the project uses.
A month in: the honest verdict
After running this setup for a month alongside Claude Code, here's the unvarnished read.
For boilerplate and lightweight refactors, Gemma 4 is plenty. About 30–40% of what I used to send to Claude Code now stays local, and my API bill reflects it.
For multi-file refactors and "go research this unfamiliar SDK and implement against it," Claude Code (Sonnet 4.6 / Opus 4.6) still wins on stability. That's not a question of which is "better" — it's a question of where each currently shines.
Where to take it next
OpenCode's real value lies in customizing agents to match your workflow. I keep a dedicated commit-message agent and a test-case-generation agent alongside the main builder, and switch between them as needed.
Define too many and you'll waste time deciding which to invoke. Start with three or four, then add more only when a clear gap appears.
Local LLM tooling moves fast — six months from now, the recommended setup will probably look different. What matters is that "a coding agent that runs locally and gets real work done" is finally a real thing, and the hybrid of cloud AI and local models is going to be the dominant pattern for a while.