Sure, trying to make due with 49 ability points can be rough if you're actually trying to pay for the 4th and 5th point. That's generally why you try to use gear with ability bonuses on it instead. It's a lot easier getting multiple skills to 5 when it costs only 6 points instead of 15 points.
But then that gear is otherwise worthless more often than not and to keep rotating gear is a different pain...especially when the inventory system is the mess it is, filters only do so much.
And that's why God invented backpacks. And also the ability to put backpacks on your skillbar.
Seriously, if I had to list the top three benefits of using invisibility to sneak through Luculla at level 3, they would be:
1. Unlocking Teller of Secrets hellaciously early
2. Unlocking Tenebrium skill hellaciously early
3. Picking up 3 Pouches, which combined with 2 backpacks and Evelyn's Backpack gives you one for each character for a standard "item menu" of food/potions/gearswap, a "quiver" for your ranger's special arrows, and one for common craft tools like hammer, axe, knife, mortar and pestle.
I sometimes wonder why those pouches aren't as available during the early part of the game. Maybe the expect you to craft your own backpacks or something (it's leather scraps plus rope, if you didn't know). Easier if you don't have to track down the ingredients, though. Plus, having both backpacks and pouches makes it harder to confuse the contents of each.