Coder runs Google's Gemma 4 26B language model at 5 tokens per second on 13-year-old dual Xeon server with no GPU using patched inference code that bypasses missing AVX2 instructions

Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU

There’s a server in my basement that has no business running a modern language model. It’s a repurposed HP StoreVirtual storage box, roughly thirteen years old, two Ivy Bridge Xeons, no GPU. It was built to hold disks, not do math. As of this week it runs Google’s Gemma 4, a 26-billion-parameter open-weights mixture-of-experts model, at about five tokens per second. Reading speed. HardwareRepurposed HP StoreVirtual: dual Xeon E5-2690 v2 (Ivy Bridge, 2013), DDR3, no GPU Instruction setsAVX1 only ...