I saw someone on Twitter claim that those PCs can be removed from the party. According to my memory of a tweet I can no longer find, you need to kill them, then ask Withers to "store their soul". Does that help at all?

Avatar art by Carly Mazur