I did a play through where I chose a different party face for each encounter. It played well.

Sure, sometimes NPCs started dialogue and I got party leader or whoever was closest, but even then it was seldom completely unpredictable.