So open worlds are bad? The limiting device is the fact that all the opponents were increasingly above your level, dialog and other warnings in the game about certain areas, along with the usual physical indicators that there may be stronger opponents ahead (fences, gates, bridges...).

No quests are urgent (there is only one timed quest, involving a merchant in Verdistis; if you accept it, it must be completed the same game day). For the castle, I completed all 3 of the quests that give you an invitation, did most of the first map and Verdistis and cleared the Rivertown sewers before entering.