What if your character joined as a follower? one that doesnt have any impact, such as if he "wasnt existing" IE specters in dark souls?
That way there would be no problems regarding the choices you did.
On the other hand, the XP would be an issue... so why not, upon leaving the game with your character, transfer the xp earned by your character to a follower of your friends game that wasnt used, but keep the drops and stuff like that...
Quest items would be directly only be lootable by the 2 main characters so that the guest cant run off with important stuff.

But this would only solve half the problem... You would get to play your own character in another persons game, you would actually get stuff for yourself when going back to your own game, but you would loose the progress made... But as we say in french, "impossible n'est pas francais", so there is always a way around...