I went to my sisters graduation at Northwestern where Stephen Colbert was the keynote speaker. I don't see a difference between the antics of Colbert and Yiannapoulos other than what tribe they represent. One is a keynote graduation speaker, one has riots that prevent them from speaking on campus.
To be honest, I don't think I can defend inviting Stephen Colbert, either. (Especially if he's in character.)
However, commencement speeches do not have a high reputation for academic rigor to start with, so I think inviting an entertainer to speak at one is sort of different from inviting a speaker with some specific experience (either academic, or political/industrial/etc.) to give a lecture.
All that said, keep that Yiannopoulos wanted to read out the names of students without legal immigration status on stage at Berkeley. That's the sort of thing I mean by "antics." I don't think Stephen Colbert does anything of the sort. His partisanship and his satirical character mean that I have trouble saying that anything of value will be lost if he stops being invited to speak at college campuses, but he seems qualitatively different from Yiannopoulos.
