The stats will be uneven anyway, as you acquire better equipment with various bonuses.
There is shrine of good and evil south of the dwarven village which will raise one of your stats by 5 (save first, it can also drop one by 5 randomly), an object which can raise you up a level instantly and another which can either raise you a level or raise the stat of your choice by 5 or 10, which more than makes up for the 3 lost skill points.
