Primary and secondary weapons you can switch between for 0/1 AP sounds like an okay idea, it would help with Ranger and Rogue builds.
It might also work for Inquisitors, if Draco 359's idea of the Inquisitor as a character whose purpose is to rapidly strip armor from enemies is right, allowing Inquisitors to carry a staff for magic armor and a physical weapon for physical armor.
If different weapons get different effects like Kalrakh's idea suggests, even warriors could switch weapons.