If you're passive and tell him to go for his dreams, he leaves you alone. He will only follow you if you are dismissive or insulting.