That's how it's written ingame, but in the actual rulebook it specifies "no weapon in the off hand". It seems like that is a mistakeful wording, but it's apparently the intended function.
The BG3 text definitely misleading because I've tested it (and actually I just tested it again right now to make sure I didn't misremember) and it adds the 2 bonus damage even with a shield like it's supposed to.
Good to know
