To use the single handed expertise skill, your off hand must be empty; if you equip a second weapon, then the dual wield expertise would apply, and if you equip a shield, the sword and shield skill would apply.