Fast travel? You mean the portal thing? Got to be at one to use it? Going to camp gives you the option of only resting and going back to where you were. I'm more worried about not being able to save.
You don't have to be at a portal to use them as long as they are discovered. Just hit the wings on the right of the mini-map.
