In D&D, using CHA for weapon attacks is a Hexblade thing, not a Pact of the Blade thing. I might have missed something about how Larian is going to implement Warlocks in release, however.