Really, I think the reason why people feel such a distinction between BG1/2 and BG3 is the speech pattern used. BG2, despite having wacky moments, uses "Fantasy English" in their speech. So even though a storekeep maybe making a joke, there's a lot of "Alas" and "Ay"s that keep it in the genre of conventional fantasy. Whereas, BG3 in the most cases much closer to modern day people (especially with the VA) - so when it's a joke, I can see how it can really take you out of it.
This is one of the aspects, not the VA, but the medieval dialogue that is not consistent throughout. I think Pathfinder WotR is even worse in terms of modern sound distracting VAs (Woljif and Lann). BG3 is more distracting in the written dialogue itself.
Other reasons are: complete lack of artistic direction in cinematics, poor writing with companions that don't behave as they are about to die from the tadpole (with exception of Lae'zel), cartoony hag and Raphael, the uninspiring Absolute thematic, a comedian vampire spawn, combat without grit/decapitations/dismemberment.
It is clear that Larian lacks a director (and writers) with vision for mature dark fantasy. They have been listening to feedback but there is only so much you can do when it is not your essence.
The original series took itself very seriously, despite the occasional humor. The tone was very different.
Edit: Also agree that the way they implemented speaking to animals is completely childish.
