We still could invite him there theoretically and not the other way around. He expresses interest and you can ask him about it(it hint that he will join), "intriguing couple" or something likethat he says and you can ask him if he is interested after. After you negotiate with drows with this knowledge in mind they could've added extra dialogue after you ask your LI to include Halsin and not the other way around? Make him invite himself as he does now if you are in poly romance with him.
Does this make sense to you?

For me personally, I would be happy if they just removed Halsin's banter proposing a threesome to SH and removed disapproval from SH/Ast if you are not in poly romance with Halsin. Does this make sense?

But also like, I think he should shut up on that matter at all, if he was put down in his poly romace offer by "If I wnted to rut with half a tonne of dumb muscle, I'd seduce a deep rothe", it indicates that player is not interested at all in any sexual activities with him(brothel, banter, etc.)