You can enter the bandit camp after you tell the LT the location, but if you do that you miss 3 quests
so my surgestion is wait until completing the quest " into the bandit's den".
first locate the camp it self. when you are scouting for the camp you will encounter jeane and filip side with jeane to get the password to enter the camp. when in the camp you can do 3 quests there.
