XP should mainly focus on quests and exploration. Else you feel to need to do fights to level up instead of cleverly avoiding them.
Why would you feel like you need to avoid them? Avoiding unnecesary fights with dialogue I can understand but actively avoiding them I dont...
And normally in dnd you dont get xp for exploring or questing either. Normally gold or some other reward is the incentive and you get thr xp from encounters. Unless the dm is using milestones or something simular, which is fine as well but I dont feel thats the approach that Larian was going for?