Keep exploring, there is stuff to do before you get the tower, so you'll have time to meet and pick your servants.
The 2 necromancers are in a cave a little south east of the totem, though you'll have to go east first around the rocks and then turn back southwest, past the enchanter Wesson.
Welcome to the forum.
