Janas-LLM runs 48GB AI models on 32GB laptop at 37 tok/s—13x faster than llama.cpp—by streaming experts from SSD; pure C, CPU or GPU compatible

GitHub - prabanta-dev/janas: 37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

37 tok/s from Qwen3-Next-80B (a 48 GB model) on a laptop with 32 GB of memory, 24 without drafts, where llama.cpp manages 2.8; reading a prompt, 62 tok/s against 16. Janas-LLM was faster than llama.cpp on every model compared on the same laptop (Debian 13 Linux, Intel Core Ultra 9 185H, its integrated Arc GPU, 32 GB, an NVMe SSD; 9-11 October 2026; the whole machine below): Model reading 1,024 tokens: Janas-LLM llama.cpp writing: Janas-LLM llama.cpp Qwen3-Next-80B-A3B (48.4 GB, more than the mem...