That's probably cause you guys didn't go INSIDE the harbor.

if you face the wall while being on the lift on the ground level, go a bit to the left. You should see a large gate.

Go inside, that's the harbor where you'll find these guys.

The Question isn't how
The Question is when
and the answer is now