Joram;
I thought the priest left to go to the church when given the unused scrying stone, and tested it there. Speaking to one of the people in the church would give you a quest to find out what happened to him, which would unlock the door to Lovis' tower dungeon. I didn't try doing the quest that way, though.
PAjdo;
No, the door is at the other end of the cave, away from where the ghost was. The top of the entrance to the hallway with the ramp leading up to the door is where the button is mounted to lower the chains. From where the ghost is, just head back towards the platforms, but go around them rather than jumping onto them, and the door should be straight ahead.
From the outside, teleport to the lake shrine, jump up a rock and you're pretty much right at the door. When I first cleared up to the main entrance to the tower, I tried jumping around the right side to get past the locked gates, didn't quite make it, and landed at the lake shrine (there were too many level 8 skeletons for me to take on at the time, so I teleporter out, and tried jumping from the left side of the main gate, which was easier).
