You very early on get the chance to find out that he is just spinning you along and all he sweet words are lies. You can then either warn him off or go along with it for the fun and sexy time because you think he is hot. Later on you can ask him if Cazador's victims really fell for his lame routine, for which he mocks you because you after all fell for it too. And why did you fall for it? Because there is no way not to, if you want to romance him. There are no dialogue options for someone who likes Astarion but thinks that his seducer mode with the cough-drop-voice is lame or too artificial. From the start, the range of emotions and stances the player character can convey towards Astarion is very narrow, so it's no surprise that it is also very narrow at the end of his story arc.
Someone mentioned that the player character in BG3 is less of an actual character and more a focus through which to observe the stories with. I think that is very true for Astarion.
