The difference lies in who is being targeted and why. A troll in an online game who repeatedly targets whoever is in the game with derogatory comments then the person isnt targeting people based on identity. Its just someone being a jerk.

What words we call things matter. I agree that racism is a bad thing, but if we cant separate between people being jerks and people being racist we just hurt our ability to fight against racism. If you try to fight against racism by figting against people who are just trying to be jerks all you are doing is signalling "hey other person who is trying to rile me up, here is something you can use that is extra effective."