maybe with mods this will be remedied, like in divinity original sin 2 where you can have everyone come with you if you want... makes for an interesting playthrough