The main reason is my character never feels like my own. Sure, there are some other little issues I have, but really, the character creation and progression really kills it for me. I feel like my character is always a template which I can't really mold into what I want.
So I wanted to play a rogue type character. When you think of a rogue, you think of sneaking in, pickpocketing the target, then double dagger-ing his liver or something. When you're in town, you think of a person who can talk their way out of every situation, swindle vendors for cheap goods, luck always on their side, etc.
Right out of the gate, a rogue should have sneaking, thievery, persuasion, lucky charm, bartering. But the way the civil abilities system works, you can't have a baseline rogue character until level 10 at the very earliest. And the way sneaking works, it's nearly useless in combat, which means you almost have to spec into something else that gives you invisibility... especially early game.
Now think about it from the perspective of a character with a backstory. If you pick Sebille, you have a history of being a rogue/spy/assassin... except you're not really any of those. You just happen to start with a few "rogue" combat skills. Which would be fine if you were a novice rogue... except you have a backstory and everything.
Don't get me wrong, the world is great, the environmental interactions are great, but if my character never really feels like the character I want to be, there's no real incentive for me to do anything other than the main storyline.