What if you had some staffs with physical damage? Like other weapons with mainly physical and a little bit of elemental damage, you could have the reverse for staffs, where they're primarily elemental, but have a little bit of physical damage as well.
You could also have a magic-less staff that perhaps has the All-In ability, like two handed swords and spears, or an equivalent physical-based melee attack.
