Took martial proficiency. I swapped to a shortsword and dagger combo and it's still grayed out.
Took off all my gear, took off my ranged weapon - off hand attack still grayed out. It's kinda driving me nuts. Is this just a technical glitch?
It may well be, and if you haven’t already it may be worth sending your save to the Larian support team to see if they can help, or at least for them to confirm it’s a bug.
Out of interest, though, do you still get the same issue if you re-drag the off-hand attack action to the toolbar from your “spell book”? If you have an off hand weapon you can use equipped, I think the icon should appear in the “Common Actions” section of the second tab of your character sheet.