Once you give your scrolls up to The Big Tree (can't remember his name) or Dreaven the enchanter (does he even use scrolls?) do you lose them forever and have to get new ones for someone else? Is this a big deal if you're going to get your battle tower but not for a few levels or so?
