thats not unresonable at all.
You get more Xp for doing more, i.e. fighting all the warriors which are not easy.
if you dont fight them its only natural to get less xp. C&C.
Thats what RPGs are supposed to be, not giving same rewards to every player for anything he does or dont do.