Why would you make an off-hand attack when you can Shove someone over the ledge or into lava, without any investment in a Feat or Fighting Style? Or drink a potion. Or use a Weapon Skill.
Do people who think this actually play the game very often?
You don't always have a ledge to throw someone off or lava to throw someone into. And even when you do, sometimes it's a bad idea because you want to get the loot.
As for potions, characters aren't spending the bulk of their time drinking potions.
Something like pommel strike is nice, but it doesn't compete with an offhand strike, certainly not after decent weapons are picked up. For instance, I'd much rather do an offhand strike with the Ritual Dagger than make a pommel strike.
Which, by the way, is part of the joy of wielding weapons. You can mix and match different types of weapons with different effects. In one hand, you can wield the Shattered Flail while you swing Xyanyde in the other. You're basically healing yourself and covering the enemy in Faerie Fire at the same time, making it easier for your companions to hit.
This is way more valuable when you're fighting the Bulette, for instance, than it is for you to be able to shove.
All this hate against shove is misplaced, although I suspect there's not that much hate as most of the negative comments seem to come from the usual suspects. In my opinion, it's a fun addition to the game. I enjoy it in those occasional moments that it's worthwhile to use.