You must find another door, lower on the tower than where you met Carlin to go in!
If you go to the Teleporter Shrine where you jump on platform and used a lever to open that first gate ..
from there go up the way above and go through all opened gates! (all gates go open with using a lever!) ... Then you comes on a door (without using any ladder!)
Success
