People ready and willing to spread violent, harmful hate can be dealt with via the laws they violate. Assult, battey, stalking, theft, etc.

The subtle side is, as always, if your speech cannot persuade the (large) majority that the opposition speech is wrong, then to dismiss the opposition is to become the oppressing minority.