Think of the most common restrictions on speech in a liberal democracy: commonly cited examples include yelling "fire!" in a crowded theater and defamatory speech. One might be able to assign rights to whole groups of people (as international and domestic law sometimes do). One of these laws may be analogous to anti-defamation laws protecting individuals against libel or slander. One way to look at hate speech is that it is defamation against whole groups. Thus if we can justify anti-defamation laws in a liberal society, we can justify anti-hate speech laws if hate speech is analogous to defamation. The most obvious retort is that they are not analogous.
Some hate speech clearly are similar to defamation such e.g., spreading the lie that Jews are involved in a global conspiracy to kill all "Aryans." That is a claim, a false assertive claim about a group of people made to defame that group and comes at considerable costs to that group and thus may be liable to analogous legal repercussions. But consider a racial slur, "nigger." Referring to someone black using this term is not, on the surface, making an assertive claim about her or her racial group and thus a fortiori, not making a false defamatory claim. So on the traditional interpretation of racial slurs, "That nigger, John was fired" would mean the same as "John was fired" or "That black man named, 'John' was fired."
But is that really the case? Some philosophers of language such as David Kaplan and Christopher Hom claim that certain nouns including racial slurs do have assertoric content. Hom's paper "The Semantics of Racial Epithets" is a classic work on this topic. For racial slurs such as "nigger," Hom argues that it is translatable to a conjunction of sentences (or propositions or whatever it is that have semantic content) that makes pejorative claims about a group or a person belonging to the group. The longer and more institutions of hate associated with that slur, the more semantically "explosive" or harmful it is because it make more implicit assertions that are packed into the slur. When used, the slur becomes "unpacked" and explodes all the (defamatory) assertions associated with that group within some society in which there is a history of institutionally associating negative stereotypes with some group to the slur.