Fortunately the pathfinding in OS games is best ever pathfinding in any game ever .
So you can literally click from one side of the island to another and have the party get there without a problem.

As for Fort Joy, it would be nice if we could atleast say something to the other prisoners after we killed every single magister there. Maybe they could decide to stay there anyway because they dont want to cause trouble.
maybe the elves that will maybe go there to get Sahela the Seer could collect them... but it would be nice if there was some solution and closure to it, even if it happens off screen.