I know some people prefer larger parties in BG3, I have to disagree about 4 being too small for a D&D game

I've played at a table of 7 and there's a lot of waiting in combat and not much roleplay! I've heard older editions used to be designed for bigger groups, but 4 is considered standard in 5e. I think this is why BG2 had a party limit of 6, and BG3 has a limit of 4.
Although you don't have the same attack AOE spells with that party assuming you haven't changed origin classes (although Spirit Guardians <3), maybe a better approach would be to use battlefield control spells, which bards are good at, to let all your martial characters beat people up more easily. If your bard is a lore bard you could always take Counterspell as a magical secret at level 6. Level 10 for other bards is a bit late.