News Score: Score the News, Sort the News, Rewrite the Headlines

Arcee AI | Trinity Large: An Open 400B Sparse MoE Model

Two months ago I wrote about why we decided to stop treating pretraining like someone else's job.At the time, Trinity Nano Preview and Trinity Mini had just released, and Trinity Large had started training. We were in the middle of our first run so big that you either laughed or got nauseous. Frankly, I felt either we’d end up with a really great base model or fall flat on our faces with a tired wallet.Little did I know, we’d get both.Here’s what we’re shipping, what surprised us, what broke, an...

Read more at arcee.ai

© News Score  score the news, sort the news, rewrite the headlines