I actually really like hard penalties as long as they make sense. Choices should have consequences. Not everyone is going to be happy with you and they will tell you about it if you do a thing that you don't like. I just think that leaving them sitting idle at a camp that shouldn't exist when there are things they could be doing should count as a thing they don't like.
I'd really love to see something like, you get to the blighted village and there are multiple different paths you can take. Everyone that's following you needs something to do, so you take your party of four to go check out the path to the north while you send Lae'zel and a few more people in your group to check out the path to the west. You meet back at the village in a few days and discuss the things you've discovered. This means a few things:
-Everyone in your group is always occupied, whether they are coming along with you or not.
-You won't be the first one to explore some areas, because you're sending other party members off to do that. They might handle things differently than you would. So it's important to decide what you want to handle personally and what you want to entrust to someone else (and who you're entrusting it to).
-If one of your party members feels strongly about going in a particular direction (e.g. Lae'zel towards the Gith patrol or Wyll towards the goblins), they can bring it up with you when you're making that decision. How you take their feelings into consideration might impact how effective they are or whether or not they're willing to keep following you. On the other hand, if you give them too much control, maybe they try to take over.
These characters may have their reservations or their insecurities, but none of them seems content to sit around and do nothing for days on end. Let's give them interesting things to do.
Ultimately, this is for another thread most likely, I want my decisions to take options away from me. I want to have to commit to a path and have there be repercussions if I change my mind. If all options are available to me at all times, the game is going to get boring and samey.