I don't think Voidwoken can climb ladders. That seemed to provide a pretty easy way of killing them in the swamp.