It's super immersion-breaking and I'm sure Larian has already run into this exact concern.
Unfortunately, the problem here is that the concept of immersion is completely detached and alien from anything related to encounters and combat, for Larian. Worse - given the comments we've seen and the jokes that get made, they (or at least Sven and by extension the team) are aware that so much of their game design is complete anathema to immersion; they simply don't care, because it's not an important consideration to them, not as much as the other things they're putting ahead of it.
Don't mind me, I'm just generally using the 'immersion' argument partly to highlight the detached absurdity between the serious effort spent on character interactions and cutscenes, while the combat may encourage you to ignore it in the same breath against certain encounters. But it's mostly to mock people I've seen who don't want to entertain something like how Solasta handles reactions because of it, when the entire combat system itself as it is currently designed already buries the concept of immersion six inches under the ground.
