Last I checked, ignorance wasn't usually a defense but I'm not a lawyer. I just know not to pretend the Keurig I bought off the back of a truck was a good deal for everyone involved.
But physical analogies to IP fall apart quickly so I'm not going to encourage people to read into that too deeply
So there's two different concepts. One is original creative output, which is copyrightable. The other is information, which is not copyrightable.
If you find something verbatim identical in a bunch of different places, you've got a strong case that it's just information, because if it were original creative output it wouldn't show up identically in multiple places.
If it turns out everyone was plagiarizing a single source, but you were unaware and took down the offending content when asked, you won't have much in the way of legal liability.
But physical analogies to IP fall apart quickly so I'm not going to encourage people to read into that too deeply