Can someone (politely) explain to me why this is actually a bad thing? Since afaik without knowing any details of "the plot" how is it possible to say that this is good or bad?
Because the plot is irrelevant.
The idea is bad from a mechanical standpoint as it will severely cripple the amount and variety of characters interaction across a single playthrough (both in general sense and even worse when tied to a game that defaults to a small party of four characters).
Whatever they'll do with the plot will be their attempt to adjust to this mechanical decision, anyway, and not the other way around.
