Quote
Meanwhile my headcanon is he just left them be because man is too prideful to go crawling after them.

Okay, you are totally entitled to your headcanon, just like I'm entitled to mine. For me it doesn't make sense that AA has no pride to actually let Tav go and not enslave them in the end, but somehow he does the opposite out of his own free will when they break up with him just a few days earlier.
Much more logical explanation to me is that he is just afraid of other companions, because it's canon they are all very worried about Tav, once Tav becomes his Spawn.

Quote
Yes it's almost like there's a scenario in which he lets the player go freely and it's relevant to him letting them go to Avernus or something. Crazy I know.

Once again, what's your point? Okay, there is such scenario, just like the path where he forces Tav to stay with him is still equally canon. I mean, he literally forces Tav to stay with him the same day he allegedly lets them go out of his own "free will".
These two events directly contradict each other. If he was really okay to grant Tav their freedom, he would have NEVER forced them to stay with him. Instead, he does it twice. At least. Considering he says: "Not this AGAIN", it's implied that Tav regularly asked him to let them go.
My explanation is perfectly reasonable and nowhere does it contradict canon.