You need to save up your leveling points to buy the higher values in magic and skills. 1 point for level 1 then 2 points for level 2, 3 for level 3 etc so it's 10 points total spend for level 4. All skilled up is a big help there as are items that boost them. By near end of beta content I got +3 on way of the warrior from items for my Battlemage last build, so it only cost me 3 points to get level 5 way of the warrior and all those lovely extra action points.

also, if you manage to find the low level dungeon run inside the town, you may dig the game a bit more.