Even more well-hidden than that. Underneath the buttons you're talking about, there's a little toggle between the words "One Handed" (which I'm realizing should be hyphenated) and Dual wield (which should be capitalized). If you set the toggle to one-handed, it won't auto-attack with your off hand.
Ha! Thanks. I thought those are descriptins for the buttons. I will test it later tonight.