Looks like you've got a lot of good feedback already. I think your struggles just come from a lack of history with basic D&D mechanics which is nothing to be ashamed of.
It's important to look at abilities / weapons to understand what stats influence them. To most of us, we just know already, and we don't think about it. But for someone new, it might get lost on them that a Wizard's Firebolt uses their INT to determine chance to hit, but a Wizard's Acid Splash does not. Instead it forces the enemy to make a DEX save vs the Wizard's DC (Difficulty Check) which happens to be influenced by their INT. This is really important because against an enemy with a really high AC the Firebolt might prove way less successful, but against an enemy with really great DEX, you probably don't want to use the Acid Splash.
Similarly, melee weapons are usually using STR or DEX for their chance to hit and damage, so if you equip a battle axe on your DEX based character with low STR, you might not realize it's using your low STR now and that's why you're missing.
All that being said, +1 magic weapons will help the classes that attack with them to hit (won't help your wizard at all, though there will be future magic items that do that, but I didn't see any in the EA yet).
Personally, I found the Cleric fairly useless given how much free healing exists with abundant potions and food. However a spell like Bless is quite powerful so that is an option. I did Ranger (forgot exact choice, I focused on bow stuff) + Laz (built as a 1h / shield tank) + Wyll (as other's have mentioned, high CHA and use Eldritch Blast) + Gale (high INT, usually using Firebolt, Acid Splash, or whatever the Ice cantrip was). Ranger had the best hit chance (I think 15% higher than the other two due to +1 magic weapon and +2 to hit with bows from a ranger bow choice).
Thank you for your reply! I'd imagine you're 100% correct. I am completely new to the DnD mechanics and literally had 0 experience with it when I started my BG3 game a few days ago. My inexperience is kinda why I decided to join up the EA team, to have a chance to learn the very basics now before doing a serious attempt for a game clear at release (cause let's be honest, if this would have been release I would probably have been toast already).

I've used the wiki as much as I can, but even though there obviously is a lot of dedicated people there - the wiki contains rather little information for someone who don't know the very basics... Which is why I turned here instead. :] But I am learning the very basic stuff, slowly but surely!
I am doing much better since I put Wyll on the bench, and followed all of the helpful tips that people gave me - I even managed to clear the entire outer goblin space (outside the building/old cathedral) with ease by engaging from the bridge rather than the door and doing some fun grease + fire combos while simply sniping down enemies approaching. Most of the small fry just died when running around in the fire.
Right now I am trying the same comp as before, but I benched Wyll for the cleric lady. I am going to try swapping her for the yellow lady (Laz, was it?) because in all honesty, she is not doing very much for me either. Like you said - health problems are usually solved by potions and I feel like I would rather have a better front-liner than a squishy cleric right now, and as I've understood it - out of our followers Laz seems to be the only viable frontline-ish option (unless you play a frontliner yourself - which I don't >.< ).