If u plan to continue until FoV then dont mind,just play further and slowly gather stat/skill books along with some additional stats form mind reading. When You reach FoV u will have enough skill and stat points to boost second stat. If u will plan ur equip well along with charms u can max out even to four stats.
I started as a ranger and had 100 dex,70 vit and rest was like low numbers. Completed most of side quests in FoV now and my stats are 100/60/100/100/100 atm so it doesnt matter in long run as long as u can respec ur skills.
