Follow the mountainside path with all the Imps...as you close in on High Hall the doorway to his cave should be on the mountain to the right.

The locked cave entrance I think is the one for the 4 Treasure Hunter quest