There is nothing you can 'invest' in Fort Joy. You got nothing at the start and there is not much to gain either, so what exactly should you invest? Even if you know were all those nice 'armors' are lying around and were chests are hidden, your equipment stays subpar for the most time of the act. So this argument is kind of bullshit.