when ever I climb up a latter or climbable surface by clicking directly on it my character automatically climbs right back down after climbing up... annoying when you are just roaming around, detrimental when in combat (as it uses up movement)

Yeah, it's annoying. My workaround is to not click on the ladder or climbing surface, but just a little past it so the character moves off of it.

Yeah that is what I have been forced to do though in some cases it's been difficult cause the camera doesn't want to move to the top of the area so I can click there :P

yes it is very annoying, specially in combat