This definitely needs to be added. It is frustrating to have to recruit and then unrecruit people just to access their inventory. One way to do this would be to put a chest next to each character at camp that contains their inventory when not in the party so you can both take things out and put things into it.