You are correct about the location of the entrance. Check out the area far below the talking tree across from the Red Hammer tribe, at the far back in the area with a few dragon skeletons around. At the cave-like entrance you need to read the Note from Maxos in your inventory to clear the ice/glass.
Once you enter the Hall of Echoes (finish off everything else first), there are some memory eaters and nests, with a barrier preventing progression until you destroy the nests.
