It seems to be running fine now, I've quickly loaded the game to see if anything had changed and swapping characters is not freezing the game at all and the summons are instantaneous without any kind of lag, gotta check how the game runs during combat.
What I find weird is that Larian has not addressed this issue directly in the patch notes, just some optimization notes that they improved performance to reach FPS. But well, I'm happy that is running great again, finally I can get back to play the game.
