There are story reasons why the security is not particularly tight. Just as you are about to enter the palace, the door unlocks automatically for you. Cazador is expecting you to bring Astarion so he can use him for the ritual. You can get extra insight into Cazador's way of thinking if you chat with him alone first and then talk to Astarion at camp. Astarion says: 'He won't throw the door open for us, he's no fool , but there may be fewer obstacles in our path if he's expecting us.' It looks like he put a modicum of enemies in your path, thinking it would be enough to best you or at least weaken you before you face him. He's very sure of himself and thinks you stand no chance against him.
About the palace - I agree, it's too small and is positioned in a ridiculous location. It was clearly meant to be in the Upper City, but since we didn't get it, they awkwardly squeezed it in. I think there was a line from Astarion saying the palace is imposing on the outside but sadly we don't see that in the game.
I never talked to Cazador though, and when he sent his spawn to my camp to retrieve him we almost killed them until he teleported them all away so he had no reason to think I'd bring him anything. I think this idea of the path being made easier could have been handled better (and may still yet in patches to come) even if it was more people or whatever you could encounter, who might step aside if you have Astarion in your party, or where you have to pass persuasion checks to convince them that you and the other other random people with Astarion are totally supposed to be there too, and who will fight on sight if you leave him in camp. Just layers to how it could go depending on who you bring with you and how you approach it. Maybe like the
Isobel fight where they try and knock Asatrion out and take him away and kill you when you enter, showing the two contradictory natures of a vampire being super arrogant but also insanely paranoid.
And yes to everything Ecc2ca said.
I found that document of the past vampire lords as well and It made me really curious about them, but of course there was nothing else. Finding documents, or a hall with their pictures, or things around that showed more of that cycle of abuse you're there (depending) to break would be fascinating and much more meaningful to Astarion's decision to break free. Not to mention, I'm just a lore hound, and I really *really* just wanna know more about whats going on here XD I would also just love to find more personal effects of the Spawn. Astarion has lived there for 200 years and I found literally nothing of his? Even if he has a negative reaction or wants to leave it all behind, it would be neat to get a glimpse of his life there in a more personal way than just what he has told us. Especially more about his relationship with the other Spawn. I don't even know if he likes/cares/hates/pities the other Spawn.
Also random nit-pick, where is this sewer entrance to the underdark they spoke of? I combed that dang 'Palace' and found no such thing, not even a door hinting that was what it was and I just couldn't go in there XD