On the second floor you need to read the mute goblin Erlking's mind, and then read the book he was thinking of in his bookcase (on the right as you enter the room, at the far back); you may need to scroll down to see the end of the list. There is a key you need after that on an abomination master on the third floor, which you may have already. Finish everything else (with a couple exceptions, described above) before entering the summoning chamber on the fourth floor.
In the derelict tunnels, you can use the 3
Talismans (there is also a talisman of the south) in the correct order at a statue in the middle of that tunnel (a well lit statue, with a plant or something on either side, in the room at the top of the stairs down to the southern branch), to give you the Talisman of the North (which has +2 Wisdom, 3 enchantment slots and a couple other random bonuses). If you want it, this must be done before taking over the battle tower.