<img src="/ubbthreads/images/graemlins/wave.gif" alt="" /> Welcome to this forum! <img src="/ubbthreads/images/graemlins/wave.gif" alt="" />
In principle it doesn't matter, however...
...when you eventually leave the castle, you directly are sent to Iona's dungeon, which is huge (and a tough one in the lower levels), so you better be level 25+.
Generally it is advised to do all Aleroth, Rivellon and Verdistis quests before entering the castle.