Advice - West Gate, hover of enemy before engaging to see their level. You can use sneak.
In town there is a lot to do to get EXP, I don't think you've touched that much on the potential there. I do know my first time in town, I was more chomping to fight vs figure it all out, it just caught me off guard.
You always can change difficulty in game at any time. Once you get your sea legs, you can bounce it back up.
