Right, but we’re not talking about a worst case scenario where models are trained primarily on their own output. We’re talking about what will happen in the long run, as future models are trained on a realistic mix of content. It’s surely an open question what will happen.
I think theres research that shows that models collapse after being trained on their own content.
https://www.nature.com/articles/s41586-024-07566-y