42x Faster Prompt Lookup Drafting in llama.cpp
42x faster prompt lookup drafting in llama.cpp
Hayder Tirmazi
[homepage]
[github]
[twitter]
This article was originally published on 2026-09-26.
TL;DR I make drafting for prompt lookup decoding in llama.cpp up to 42x faster while using up to 2.6x less memory through
a set of simple performance optimizations largely based on the work of
Daniel Lemire and Martin Ankerl.
Update: Daniel Lemire sent in a PR
that makes prompt lookup drafting upto 4.2x faster on top of my original optimizations. His wo...
Read more at jadidbourbaki.github.io