So I found what i think may be an issue when you have a downed party member. You cannot travel to your camp if one of your party members is dead. This kinda makes the Skeleton guy that you get that offers to resurrect the dead for you for 200 gold each kinda pointless as he cant be used at all.