I think it's important to consider why the game is structured this way. As I see it, being able to frequently return to camp is how Larian wants the game to be played.
A lot of the plot progression is tied to the camp. That's why there's only one short rest and the combat's on the more difficult side. It's all structured to encourage you to go to camp frequently. That in turn moves a lot of these plot elements forward.
And we'll have even more party members in the final game, so they'll want us to juggle our team around frequently (and a lot more talking) before we settle on our final 3 teammates.