In-game efficiency aside, I don't think it would "break the game", but conceptually fighter-class weapons are supposed to be for fighters, not spellcasters. It's as simply as that.
I have no objection to one or two physical weapons specifically designed for spellcasters, though.
Last edited by Try2Handing; 12/06/18 05:46 PM.