Been there.
I also had a combat at lv2 where the bad guys crit 3/4 hits to start the fight and I had 2 party members down and the other 2 half dead.
I do agree that having more enemies or giving enemies more abilities is a good way to increase the difficulty though.
