You need to enter using a different entrance. There is a crumbled bridge with a ladder to get up it. The large iron gates can be opened when inside the tower.
There will be an item you should (hopefully) find that will teleport you into the lower chamber to unlock the bottom door from inside.
