Strong +1 that climbing ladders should require movement. One of the main benefits of high ground should be that it takes a while for enemies to reach you. My Misty Step shouldn't be countered by your ability to essentially teleport up a ladder. I'd be fine if ladders costed normal movement, but would prefer if they (and all climbing surfaces) took double movement. And if you don't make it up, you end your turn on the ladder.
It's probably hard to implement staying halfway on the ladder. So I guess it would be fine if you could climb 20+ meter ladders in a single turn but IMHO then at least your turn should be finished.
Another possibility would be to make climbing ladders a full action. So that you at least lose your attack for one turn.
