It's not that hard to understand, actually. It's simply finding joy in overcoming challenges. From my own experience, someone saying "omg this boss is so **** broken wtf were they thinking?" may be just a simple outburst and that doesn't necessarily mean they're not having fun or that they've actually given up on trying to overcome the challenge. Ultimately, overcoming harsh challenges more often than not involves some degree of "bashing one's head against a wall". It can't be helped.
Clearly, lowering the difficulty is one approach. But in the majority of cases, there exist other answers beside "lowering the difficulty". It may be perceived as "ego", but, I believe in most cases, it is really the belief that "it can be done", and not giving up. And finding the answer after wracking your brain, is fun. I have no doubt that I can finish the game on this difficulty. I know it can be done, I know the answer exists. The fun here is to find that answer, the "how". How can it be done? It's not important that "omg I just want to win this", because I know I will win it, eventually.
I'm playing WotR blind on Hard, and just earlier I cleared Wintersun with a L11 party. It was brutal. Not even "meh it wasn't that hard". I couldn't beat several fights when I first got there. But I knew it was possible; I just didn't have the right party setup. I just went back to base, respec'd my Mutation Fighter into a Magus, bought Jinx, formed a different party, then came back for the second round. And most everything worked out as I suspected it would. You could say I found my answer to a problem I couldn't solve before, even though other players may have even simpler answers. In fact, the fight with the tree was really, really close. My anticipation was quite off and I was forced to improvise and use everything I possibly could from my inventory - all the Alchemist Fire and low level scrolls that had been collecting dust. In my last attack I was lucky to land the attack and roll just high enough for a SR bypass check, and wrapped up the fight before everything went to shit. If I had simply lowered the difficulty, I would never have had so much fun.
This is not to sound elitist or anything. I'm just trying to explain what it's like in my case, because admittedly I'm one of those "unwilling to lower the difficulty". The summary here is that, yes, I may get frustrated sometimes, I may get mad sometimes, but it's the belief that it can be done that keeps me going, and knowing that finding the answer in the end will feel great.
As for the "unfun" part... Personally, if I ever say something is "unfun", then it would be more about the design, than the actual difficulty. I can lower the difficulty and that still wouldn't make certain encounters any more "fun". They're still unfun. They're just less of a bother.
"We make our choices and take what comes and the rest is void."
