agreed, one of my main cristicisms aswell.
do it just like in kingmaker -> highest modifier in the party applies. thats why you have a party, so not everybody needs to be the "i care for animals" or "i am fkin handsome" dude.
Perhaps block certain characters from a Dialogue option, if it would be against their believes. Like if some option goes directly against the teachings of shar, shadowheart may not make that roll, so the next best is taken.
