Yeah, I couldn't agree more. I usually play as the more dialogue-oriented type of character and it's not fun to think I'm just gimping myself by doing that. To be fair, there is an argument that it doesn't make a lot of sense for you to grow as a powerful fighter or mage thanks to all those charming conversations you had while avoiding actually fighting. But at the end of the day it's a game, and one based quite heavily on exp gain, so you can't really screw the player out of that resource because they're roleplaying in an RPG.