I just checked the advanced options again.
There's an option that fills map positions with other players and it's on by default.
Disable it and only those players that entered the map will be present (not sure about team members, though).

By the way, I could reproduce the "auto-draw" battle. If anyone's interested, I've got a save just before the fight.