Got the same issue two days ago. Due to the bug with ladder, he fell behind the scene. He can attack you but the opposite is not true ... You can move away from his location and he would finally "magically" return on the normal playground. I advise you to destroy the ladder to avoid the situation twice ....
Bug reported to Larian dev team.
