According to 5e rules*, you add your ability modifier to an off-hand attack roll but
not to damage.
https://roll20.net/compendium/dnd5e/Combat#toc_6 look for "Two-Weapon Fighting"
The fighting style, confusingly also named "Two-Weapon Fighting" allows you to add your ability modifier to the damage of the second (off-hand) attack.
https://www.dndbeyond.com/classes/fighter*Larian might implement two-weapon fighting to work differently in BG3, who knows?
