It is never too late to get the sigil? since you need it to complete the game. my guess is u tried to fly there by sentinel island, which is wrong.
Once you obtain your battle tower, you can teleport to the Grand Knight Shrine. There should be waterfall with a huge statue of a soldier "standing on" it.
Morph to a dragon and fly past that statue, heading away from the waterfall, down the valley, and u will reach broken valley, you can follow your map coordinates from there.
