Shoving people off sizable cliffs, let's say ladder height, only tends to deal like 5 damage at most, and they just get up, hardly having lost movement. That's crazy good for a bonus action, but for an action not amazing. It needs to feel rewarding to sacrifice attack and take these opportunities, but not be able to shove everyone with anyone and have them fly like a rocket