A lot of the posts are missing the problem.
The problem is your team is essentially 3 physical damage dealers and 1 magic damage dealer. Though this is manageable, its not optimal if you want your team to do magic damage. You need to consider that this game is much easier when your entire team does one damage type (physical or magic). Mixing the two actually makes you weaker in a fight. Also, physical damage greatly out-deals magic damage (its not even close) however magic does have more utility if you can punch through magic armor, which also out-scales physical armor by end game. So not only does magic do less damage, but you also have to deal with more defense.
That does not mean magic is bad but as you can probably guess, you would be doing much better if two or more of your characters were int based magic damage dealers. As for single magic dealer focus, I dont recommend it. Being versatile is actually a good choice because you can always hit an enemies weakest point. That alone will more than make up for any sort of focus you attempt to do. Its true that every now and then you will put up big numbers if you are focus but you will also do poorly when faced with any sort of resistance. In this game, elemental resists are very common and so are immunities. Specialization is not a good idea and further, your bread and butter as a caster is control, not damage. If you want a damage type team, just go all physical on everyone. There is no point to mixing. There is only one enemy in the entire game that poses a problem to physical damage dealers and she is optional and not important to the story at all, and you can still hit her at about 50% rate so she is not a big problem.