The only benefit is if you have the dueling trait for +2 damage, otherwise how much not having an offhand ends up mattering depends solely on your build and what you want to achieve with the character.
For example; I have a character with certain limitations imposed upon her to fully maintain specifically envisioned roleplay, yet despite the limitations she is incredibly versatile and powerful enough that they do not end up hindering her combat efficiency in any meaningful way. Some of these limitations are that she may only wear clothes/robes, exclusively use a specific longsword for the entire game and never ever use any bows.
So there is no benefit in doing so, but it also may not necessarily become a hindrance in any way that matters. It all depends on your build and what you want from your character.
