Ah. You can use the Teleport gloves to send someone up to the ledge near Mona's tent, then they can lower the ladder there. That will get you access to the fort. You can also use the gloves near the beach where the rainbow and chest are by the hidden area on the island to get down to the ground level, where there's a hole to get you into the prison's unlocked cell, and form there you can sneak to get the key, which gets you into the prison and from there you can reach the ladder up top.
You don't need Lockpicking 1 or Sneak 1 to snag the key on the stool near the houndmaster, Sneak 0 will work to let you get the key.
Fighting the dogs and killing silent monks doesn't turn them hostile. The big danger with the silent monks might be if one chooses to buff a nearby silent monk, bringing them into the fight.
Also if you have freed Amryo, he should have marked an entrance to the prison on your map so you can get in that way.
