Typically, once you enter the Goblin Camp the first time, you see Volo...go interact and get Volo removed back to his cage. Once you get inside the defiled temple of Selune, find Volo, set him free but make sure to get him to meet you at your camp. That will nullify what just occurred with your playthrough.
