A work around if this bothers you is fast traveling out of the camp, i hate camp scumming it really takes the fun out of the game if you can restore all your stats at a moments notice between every fight.