Just a quick point of order? Can people PLEASE stop saying that dumb jokey humor doesn't "feel like DnD". Anyone that says that clearly hasn't played much pen and paper (stupid jokes are the norm).
This is going to depend on your group. I have usually played with people who take things more seriously, which I prefer (there is still some joking around obviously but not to the point of stupidity.)
Even the original Baldurs Gate games had plenty of silly humour, that broke the 4th wall or poked fun at the game itself. Especially 1 had silly encounters hidden here and there on the world map. I think a main thing with why it feels strange is that it happens so much. If every other encounter is a Monty Python sketch it loses its charm quickly and makes it harder to immerse, where if it is a rare thing that happens now and then makes for a fun tongue in cheek moment. It is like all humour basically: timing is important. And balancing it with the serious parts. It might make it feel more like finding a fun easter egg (perhaps one of many) rather than feeling like you are basically in a Monty Python movie.
I feel the first BG had rather too much of this stuff and this probably is part of why I couldn't really get into it. A bit of silliness is fine but too much of it irritates me and feels out of place, especially in situations that are meant to be more serious. So far BG3 seems to do a lot better about this.
