Nope, you don't have to take them with you - but it is important to click on them as soon as you have found them - if the message pops up: acess to BF xxx => you've unlocked the entrance.