Rich Sutton, recent winner of Turing Award, is well-known for his 2019 unpublished essay called The Bitter Lesson that arguably foresaw the rise of extra-large language models. The central thesis (which I have always felt was overstated) was that progress on AI has always come from scaling, and never from hand engineering. Advocates of LLM scaling love the essay, and refer to it often.Their favorite part might be this line, which could be seen as the rally cry of the LLM revolution.One thing th...