When you entered Fort joy, did you do it by going up the ladder and playing cards with the gang at the table? If you engage in that card game and do not fight them they will steal all your items on your controlled character.