I have found that the game is sluggish whenever I move/transition to an outdoor area from a building/cave/dungeon. The workaround I have found is to alt-tab out of the game and then alt-tab back in a few seconds later.