Hold the left trigger to bring up the character / party selection UI. From there, the left stick can select a character and hit triangle to split them from the group or rejoin a group. If you have a full party and want to split them into 2 groups of 2, you can hit the X button for the 2 characters you want to split off, and then triangle to split; similarly, if you want to join individual characters, you can select them first and then join.
You may want to split one character off from a group to scout ahead or get through an area using invisibility or sneaking, and then have the reset of the group join up by teleporting to that character. Separating the characters also allows you to position party members before a fight, etc.