Yeah the only reason you equip weapons is for passive boni, which kinda sucks.
Skin Graft and the massive damage helps you not needing to autoattack, but it's still sometimes weird.