Then I think it is maybe a matter of your gameplay prefernces vs how you level up. Do you focus on attributes that support your preferred style?
If figher (I understand this is the case) then focus mainly on vitality, strenght, dexterity, somewhat depending on specialization.
Choose a fighting style you adhere to: two handed, dual wield, shield and sword, and focus accordingly.
X box must surely have difficulty level too, though.