The female assassin in the cave is not the one captured by the thieves' guild. Visit the cellar under Leptine's shop in Verdistis and ask the bartender for special treatment if you want to see Penumbra again (assuming you saved Corinna the first time you entered her house).
Just walk into Dr Elrath's house; he will always kick you out. There is a key south east of his house you can use to enter, or the door can be Lockpicked (level 2 IIRC) until you confront him with certain evidence.
The orb in Kistandalius' shop is just decoration; the only things of value are a spell book on the table, and a potentially very good dagger in the locked chest. There is a brief opportunity to get into the back unnoticed when you pay the minstrel outside to recite some poetry. Other than that, you would have to kill Kistandalius to enter the back of his shop; even out of sight he teleports you away.
You can bring both teleporter stones with you, but if you don't you can find replacements hidden near the start of the wastelands.
You can not rescue the cow thief; even opening the cell he will not leave. He gets arrested whether you turn him in or not.
It would have been nice if the background music changed after clicking on the piano. There was a minstrel in one of the old Ultima games (IIRC) that you could ask to play a song and the background music for the town would change. I preferred that song, so always talked to him when there.
The dragon amulet may be named that, but it is not part of the dragon set.
