Near the end of the channel south of the harbour, before you enter Rivertown Gorge / Kali's flying fortress, there is a fair sized rock in the middle of the river with the chest on it. There is one wrecked boat nearby (presumably the chest with the gem was moved off the ship itself, to the nearest safe location), and several more in that section of the channel.
