Ooh, I have a good idea! Each Wizard tower could have a small selection of randomly determined cards you could trade for at a 4 for 1 ratio (weighted towards card rank), say 2 or 3 that changed every turn. More Wizard Towers means more of a selection.
This also stops the frustration of trading 4 worthless cards for 1 worthless card. It would take a bit of coding but would be easier to balance (heck the cards should ALREADY be balanced according to rank, even 1 for 1 trades wouldn't be gamebreaking). As far as the AI goes you'd just have to feed it a list of optimal and sub-optimal cards and it would trade in whenever it had a couple of redundant ones (unit acquisition and anything from the Wizard Tower itself).
I'd still prefer the Wizard Tower to generate actually useful cards but this would fix some other issues with what could be a lot less work.