Old post, but I definitely agree about the feeling lost issue.
Basically, I recommend that Vissar approach you almost immediately after the Dallis encounter at the beginning of the game and basically say "Dallis has been getting increasingly more violent. Everyone is on edge. We need to find a way to escape."
Or have Vissar literally ask you what your goal is now that you're at Fort Joy, allowing you to respond with a multiple choice thing like "I plan to escape this place as soon as I can" or "I'm going to ask around town, and figure out what is going on here". Then Vissar can specifically tell you to talk to people around town to get some leads.
Because that's what is missing. It doesn't tell you that you should wander around and talk to everyone. It just assumes you already know that, because you're a gamer. But it would give the game more direction if an NPC literally tells you to talk to people around town.
