You can change the difficulty in the middle of the game, can't you?

To the OP: I would suggest start on normal if it is such a big issue for you. Try the fights on normal. And you feel that a fight is a little bit too challenging, then change the difficulty to easy just for this one fight.