Astarion will leave if you do enough things that displease him. He first warns you taht you suck and then he just leaves.