I'm not sure how did you miss that other people "Pay attention to their surroundings" as well, which is how they go through fights.
The problem becomes precisely that more often than not the area you need to "pay attention to" covers a good portion of the current arena, with the AI being overly-enthusiastic about using that juicy bonus action shove at any give chance.
In a condition of normality a player SHOULD worry at most of not standing on a brink of a precipice/trap/lava pit. And in any other case a shove should just push them away or prone on the ground.
Not here. Here your "danger zone extends for meters around you. Being "shoved down a hole" is not something that could happen if you stand on its brink. It's something that can happen if you stand anywhere in its general proximity.
