If you finish the quest to restore the doll you do loose it, but you can avoid completing the quest. I don't know if you would get the skill points back (haven't quite got there yet). You could always hex edit to restore any lost skill points.