True, quite often the companions in camp are omniscient about a lot of things. I think for me Halsin's banter for your relationship with Astarion or Shadowheart stands out in this context in particular is because he's intruding on an intimate moment and in Shadowheart's case even trying to invite himself. This can be fine if you're in a romance triangle with them of course, but not when you've already told him to stay out of it. Should be an easy issue fix though, if the triggers for those party banters are just completely blocked after rejecting Halsin.
And yeah, the really rude response is the only way to directly shut down the whole conversation, even though the following party banter and drow twin situations will still pretend as if you had consented to romancing him sadly. All his romance flags really need to be fixed, but especially his act 3 romance stuff needs a clear, polite cut-off point that just works and is consistent for the entire rest of the game.