When I first began the game I kept getting my butt handed to me because I failed to understand the environmental mechanics.
Once you learn how to handle fire/poison/oil/water/lightning you'll be fine. You can turn the tide of an impossible battle with a well placed shot of lightning on a pool of blood, or oil around a campfire, or using fire to set alight poison everywhere.
I actually found early game quite fun. It's mid-late game when you get epic spells/skills that it tends to drag on for me as much of the challenge is gone.