You need all 4 treasure maps in your inventory to get into the treasure cave.
There is one in the Dwarven Halls, one in the south west area of Rivertown, one in the Verdistis sewers and you can get one from a merchant in Verdistis if you have one with you (through a speech option).
Silver Armour and Treasure Maps 
