Yeah, I think the difference in question is between being "driven off" because you can't stand to listen to somebody else's words, and being "kicked off" aka banned. The first allows free speech, the second prevents it.
Which, interestingly enough, is the same pattern that real school cliques use to control their social environment. How many people have been hounded by a clique at some point as kids, only to throw the first punch out of sheer frustration, and then gotten into trouble with adult authority figures because of it? A ban's basically the same thing, so it makes sense that people get freaked over the possibility.