LLMs are not superhuman at reading and writing neural network weights
A recent paper argues that frontier LLMs can work with neural network weights directly. This seems superhuman, but it isn’t.
The paper shows that GPT-6 Astra can take a small neural network and interact with its weights directly, “well past what we’d expect from humans or conventional algorithms”. Specifically, Astra was able to identify what games a small network1 was trained to play just by looking at the weights, and even tweak the weights itself. The most impressive result is Astra significa...
Read more at seangoedecke.com