You can always unlearn your skills to reclaim the skill points that go into the skills from mindreads from FoV, like Charm, Master Herbalist, etc.

My Divinity II: Ego Draconis FAQ/Guide
My Divinity II: The Dragon Knight Saga FAQ/Guide