I don't really like combat noises and spell casting either though. Never understood why we have to screech spells at the top of our lungs when casting, pretty sure vocalising can be at any volume. Most combat noises I find to be immersion breaking as well, especially for stealthy characters. Non stealthy types I can accept a bit of noise from, although they often end up sounding constipated to me.
My thought is that "Hya!" and "Rrra!" and other such sounds are a far more acceptable use of voice than your character deciding for you what it says or thinks. I dislike when the narrator does that for exactly the same reason. "You see something and it disgusts you." No, it doesn't, my character thinks that's cool, don't tell me what my character thinks. When you choose a voice in character creation, <None> should be an option. Beyond that, your voice should simply determine the tone and demeanor of your character's grunting and such in combat.
As for casting spells, vocal components to spells never really made sense to me except as a means to nerf magic casters by making it possible to prevent spell casting by silencing them. To that end, I imagine the volume of the spell, or the effort spent to vocalize the spell, might translate to the power of the spell. Speaking softly, you might also mumble, or slur your speech, weakening or ruining the spell. If the volume doesn't matter, silencing a caster would have no effect, because they could simply move their lips the right way and succeed in the cast anyway. That's a somatic component, not vocal.
So long as the game says that vocal components are a thing for some spells, it makes sense that you'd want to hear your character say the spell when you cast it. Does that mean it needs to be ridiculously loud and post-processed with weird magical reverb? No.