Whilst not requiring a roll I'd like to see climbing ladders etc having a bigger impact on your movement that currently - it should take a lot more to climb a ladder than it does to run across open ground, which at the moment I dont think it does.
Unless you're directly threatened in combat, a ladder shouldn't require a roll. It could be cool to have an Athletics check if you're threatened, with the possibility of either provoking an attack of opportunity on a fail or falling off on a critical fail of 1.