News Score: Score the News, Sort the News, Rewrite the Headlines

Can I Buy Your KV Cache?

View PDF HTML (experimental) Abstract:Right now, across the world, AI agents are repeating the same absurd act: to read one document, they each recompute it from scratch. Every agent re-runs prefill, the most compute-intensive step a large model takes, over identical text, only to rebuild a key-value (KV) cache identical to the one the agent before it just built. The same answer, computed a million times. We make a proposal that is almost offensively simple: compute it once. Let a publisher prec...

Read more at arxiv.org

© News Score  score the news, sort the news, rewrite the headlines