That was my thought as well, but when I was switching between them to test that, I had a hard time telling if single-handed was really faster.
It's not very noticeable unless you are fighting some monsters. Then you can see their health going down quicker.
