OF course you COULD do other stuff first. But in the end, you'll see that enemies are too strong for your party or not balanced to your party elsewhere. So following the White Witch quest just seems to be the next reasonable step...
It brings back memories of stumbling into the Basilisks or Ankhegs @ 2nd or 3rd level in Baldur's Gate. Surprise!
