Cazador obviously is a victim of last minute content cuts. I bet his plot and maybe even bossfight was going to be more interesting. According to dataminers

he was going to be important political figure in the upper city, give you quests and even optionally help you fight the absolute. But after they cut the upper city, they obviously had to quickly rearrange his plot, so Astarion could have a closure to his quest. Probably that's why his "palace" looks like city walls from the outside - they had to find him a new location in the lower city.