He's a dual-wielder pact blade but if he wears no armor he can use his mage armor equivalent power, so I keep him in clothing only. My main has the 18 Dex gloves!
I mean, then use the Dex clothing and see, if that gives you enough armor. You find pretty good armor later on, even one, that lets you keep the Dex bonus ( and he still has a +3 without the robe), so wear the robe until then. The armor I'm talking about is medium, but if I remember correctly, bladelock gets medium armor proficiency. If not, maybe give him that feat later.