For future reference, if you do another playthrough: You need to sleep with him in act 1 to start the romance and then have the confession scene in act 2 to be able to carry the romance into act 3. If you only do the deed but don't show much care for him later, he will break up with Tav when you select this option (it's only available if you had no confirmation of deepening the relationship) after leaving the shadowy lands:
