So, which part of them is innocent? Hired killers for money who kill "problems" for whomever they deem to be unworthy to live. That is who the Gur are.
The Gur were never innocent. They were monster hunters and didn't ask questions about whether or not if you were in fact a monster who did something outside its nature.
Did Astarion want to do what his master bidded? Did he have a choice? They KNOW Astarion did not have the option to refuse Cazador's orders.
In my opinion, they are rude and judgmental for their OWN reasons & beliefs. If you do not want to help them with their war, you become another "problem" and attack us for their own judgemental beliefs.
So since we did not participate in their beliefs and were rude going about their asking, the Gur simply attacked the party and we simply defended ourselves.