Not to mention that you can say the same of man-at-arms, too. Any turn you spend 'just attacking' is suboptimal, since you could have been flurrying, or whirlwinding.
Really not true. Basic attack is actually pretty solid and surprisingly strong in many situations.
I've been doing a lot of melee this playthrough, and how it pretty much works is that you have a pretty strong pressure to use cooldowns as movement bypassers. I'm never, ever annoyed by the CD on my Whirlwind, I don't even really use it that much (about as often as I'd use Mass Weakness on a caster -- definitely not "never," but it's not a common occurrence). However, Battering Ram and Tactical Retreat (it never fumbles so you don't need Dex) get used early and often, and in some cases it sucks being on cooldown for those and not being able to use them. When you're in close, yeah, Flurry, but if Flurry is on CD normal attacks tend to do the job just about as well; Flurry is normally overkill, a lot of the time the fourth attack don't even hit because the enemy is dead already.
Battering Ram is so much better than Whirlwind. The knockdown chance easily makes up for the damage difference, there is built-in mobility, it's also AoE, and if you're not
healing off of burning ground as a warrior late-game you are doing it wrong. It's my legitimate pick for most powerful skill in the game; it doesn't need to synergize with other things, it does it all by itself.
The point is, I do tend to use a lot of these movement skills early in fights, which is admittedly a form of cooldown spam. But once I get the melees next to their targets, I'm fully content to just beat them down with normal attacks; those do the trick just fine.
I feel you're simply underestimating how much damage a two-handed sword actually does. It's ridiculous. And it's not something ridiculous you can throw out every turn; it's something ridiculous you can throw out
multiple times per turn.
