You do notice the difference, subtle though it may me. I went from easy to hard after having finished the game and you notice how a simple skeleton down in the Aleroth dungeons can actually take a nice chunk out of your character's health. It is certainly more challenging.

I am in blood
Stepp'd in so far, that, should I wade no more,
Returning were as tedious as go o'er.