I was messing around again with different dialogue options and I know when you asked about whether he had family/friends in Act 2 and reject his whole "My heart does not stir lightly, but it does now" speech he doesn't believe you at first either. I think he says "Truly? I was so sure we had a connection. You seemed more than curious to know about my current love life- for one."

When you ask about the family/friends thing, and he doesn't answer about having a lover or not, you can confront him about like "you didn't mentioned having a lover though" or just move the conversation to something else. I didn't ask about the lover's part again so not sure why that dialogue was said from him. But then again I don't think there are much options from his dialogue tree node thing to break down what was said or not.

Also below is off topic again but it's just my experience of other characters coming onto my Tav out of nowhere and not taking the rejection well

Astarion just flat out said "I know you like me, too" in Act 1 when we barely reached the underdark but I never went with any flirty dialogue with him, so not sure if that was a bug or maybe something casual (like asking Halsin's age) triggered him coming on to my character? And Gale still brings up the rejection. I think few quests later I went up to him and he said something like "It's always a pleasure sharing a moment with you, even though not the kind I first had hoped." Or something implying he wanted something intimate/romantic but since my Tav said no well yeah. I don't know if Halsin does something like Gale does though, like bringing up the rejection again.