Well, you should look at perks first, you can buy some only if your ability at 5-base and ups from gear not count.
but ups from traits count ... for example with 4 base willpower and trait "Independent" you can buy "Comeback Kid", which normally needs 5 willpower
But you are not as limited as it seems:
After collecting 5 stones (star stones and blood stones count) you unlock a vendor at homestead (Hall of Secrets), which sells a book which increases attribute points by 1 and a book which increases ability points by 3. You can rebuy these books every level-up. If you need every point, try to delay some side quests to reduce level-ups until you get all 5 stones in Cyseal.
After spending your points on your wanted skills, don't waste the rest on less used skills, because:
After collecting 7 stones (star stones and blood stones count) you unlock a npc at homestead (Hall of Darkness), where you can convert ability points into attribute points (ratio is 5:1). Also you can convert talent points into ability points (ratio 1:10). So don't waste your points on "All Skilled Up" (adds 2 ability points) or "Bigger and Better" (adds 1 attribute point), if you have planned this, because you can can convert them into 10 ability points or 2 attribute points.
