Yeah I'm not taking that risk. Especially on my first run
I mean if you really are that worried about it you could just start as a Barbarian and then respec at the earlier possible opportunity into whatever you actually want to play as...which should be pretty early in act 1. But I really wouldn't worry that much about it...you most likely won't be able to optimize all choices for whatever you want on your first playthrough and you don't even know what all the endings are anyway. Plus some failed rolls here or there will probably derail some of your plans anyway. First playthrough should mostly be for exploration and seeing what story and ending you end up getting...then you can start wondering what if I did this other thing, or made that other choice, etc. for future playthroughs.