News Score: Score the News, Sort the News, Rewrite the Headlines

LFM2-24B-A2B: Scaling Up the LFM2 Architecture | Liquid AI

Today, we release an early checkpoint of LFM2-24B-A2B, our largest LFM2 model. This sparse Mixture of Experts (MoE) model has 24 billion total parameters with 2 billion active per token, showing that the LFM2 architecture scales effectively to larger sizes.With this release, the LFM2 family spans nearly two orders of magnitude: from LFM2-350M to LFM2-24B-A2B. Each step up in scale has brought consistent quality gains on standard benchmarks. We designed LFM2-24B-A2B to fit in 32GB of RAM, making ...

Read more at liquid.ai

© News Score  score the news, sort the news, rewrite the headlines