Sorry, but do you use the "assign characters" button and it is not working properly? It is not really clear from your post... Whenever my friend joins my game, I still control the three other characters but using the "assign characters" button I can reform the party as I wish to without any problems.
