They're all victims, he wasn't into any of them. And when he could still be bothered to choose, he says he tried to choose "beautiful people", not "beautiful men." But it's a fact that men would be easier to pick up the way he did was doing it.
As for remembering 2 out of 1000 victims, he remembers more. When you find them in Cazador's basement he says he remembers all of those faces. We concentrate on Sebastian because Sebastian is the one who talks to us. The actual problem here is that Cazador's mansion in half assed and the devs just threw a few npc models into one little cell and called it a 1000 spawn.
Bi and pan also isn't 50/50 irl. Having some preference for men doesn't mean he's meant to not want women.