I would go with skill points if you are lower than lvl 30 although later on you end up with too many. Btw the herbs you get get from the fortress are quests items and can only be used at the fountain.