It would be nice to be able to change the difficulty level more than once. I started the game at the action level, found it a little easy, then changed it to the highest level. I would like to set it one lower, but that option is not available.
You can only go up. So if you start on the easiest setting, you could change it three times. Not sure of the official reason for this, but it does stop people going up to Hardcore when they level up so they can get the bonus skill points.