I just tested dual wielding with a fighter, one without the dual wielding fighting style and then one with the dual wielding fighting style.
Neither of the two fighters had the ability modifier added to the attack roll of the offhand attack.
Only the fighter with the dual wielding fighting style got the ability modifier added to the damage of the offhand attack.
Whether this is intended or not is a different matter and I can't speak for Larian regarding that!