1. Introduction This blog post walks through how to build a small-scale distributed inference cluster using AMD’s Ryzen™ AI Max+ AI PC platform and run a one trillion-parameter class Large Language Model using llama.cpp RPC. A four-node cluster of Framework Desktop systems is used to demonstrate distributed local inference of the state-of-the-art one trillion-parameter Kimi K2.5 open-source model. Kimi K2.5 is Moonshot AI’s most advanced open reasoning model to date, positioned as a state-of-the...