I just don't see Astarion being evil
Evil or not, you can tell by the actions they approve or disapprove.
Astarion is chaotic evil, that's how I see it. He approves of killing a squirrel, approves of killing a child for stealing, approves of betraying the tieflings (many others bad things). At a party among corpses, you get his approval, and he is much happier than at a tiefling party.
Yes, he doesn't leave you if you save them, but maybe just because he's using you. Leaving is not profitable. Also, he does not approve of saving the tieflings, or any other actions close to this. Does that mean anything to you? It seems to me that the developers should add more 'disapprove reactions' so that people understand that Astarion is evil character.
He's also xenophobic...
Even if he has a traumatic past and a fear of a master, it doesn't change anything. He kill for fun, it cant be good or neutrally.
We don't have much information, he may be unhappy travel with you, but still stay to use you.
Also you don't need to have great intentions to be evil.