You do not need to go to the fjords first, but it is a bit of a waste if you mindread to go up a level but were almost there anyway. When I first went to the island I only needed 3k experience to get to level 16, so reloaded and went to fjords (saving each time I leveled, so I could reload and head to the island at the very start of whatever the highest level I could reach).

In the room at the start of the island, from the pedestal with the welcome message, head down the corridor, go left or right around a wall and exit out the door on the other side (if the wall wasn't there you could just run straight to the door from the pedestal).