AI models collapse when trained on recursively generated data
This study shows that, predictably, generative AIs or LLMs tend to decline in quality as they start feeding on content that was generated by other LLMs instead of humans.
Considering much of the web is now getting polluted by this LLM-generated slop and the web is a big source of data for their training, it seems that future models will likely regress in quality. Doesn’t seem like a very sustainable model, does it?
