You are right it is not working as good as it could be ( one of the problems the low AP cost has ) but Wand attacks only costing 1 AP while having a second weapon that gives you a bonus ( even if it is really small ) would make wand + sword > wand + wand. I dont think the game should be balanced for exotic builds ( wand+sword. sword + nothing ) but for normal builds ( wand+wand / staff / etc. ).
First of all, wand+wand is far more exotic than a sword+empty, while a wand+sword happens even in the very much mainstream Harry Potter.
Second, a major fantasy for a battle mage character is a melee weapon in one hand, with the other hand casting spells, be it empty or with an implement.
Finally, mechanical consistency demands that either both weapons contribute to an attack(as per proper dual-wield), or that the inactive one doesn't incur the cost.
It should absolutely not be done on a hand-full/hand-empty basis: if a weapon cannot be made active, it should be unequippable.
Obviously I'd prefer the combination be supported, rather than it be denied.