Seriously you only need to be level 3 to start on the fort itself.

It is tough, don't get me wrong, but 4 isn't the minimum it is the "Safe level"