and from what extent is it a negative for the society as a whole, instead of a positive

Like I said, criminalize an act not mere words. If someone is disturbing a space meant for public - that could be a punishable act in itself.

Freedom of speech isn’t as important as preservation of property rights. In my home, you don’t have freedom of speech. I will kick you out if you say something that upsets me. In public, who gets to determine who has to leave or stay?