The Scientific Literature is Poisonous to LLMs
We can think of no corpus of text more toxic to the training of useful LLMs than the 21st century scientific literature. Imagining a two-by-two matrix with the axes honest vs. dishonest and right vs. wrong, the scientific literature is splashed haphazardly across all four boxes. Papers often belong in multiple quadrants at once, and not always because of the contributions of different authors. Even worse, papers in any quadrant might deliberately masquerade as work in any other quadrant. Conside...
Read more at reinvent.science