I like your first suggestion, it would make it easier to manage follower inventory/abilities and really doesn't have much of an impact on actual gameplay. The only thing this might impact is if you take a long rest in an unsafe area and the camp gets attacked, you would then have more followers under your control than normal, but I don't really think that's such a huge issue.
