I couldn't remember if you could unseal the door to the Hall of Echoes and complete that quest before actually entering, but I had a recent save, so tried that on the way back from the Gremory chest. It turns out you get the experience as you enter, so I decided to check out the Hall of Echoes, even though I had a couple minor things left to do.
In principle the changes to the memory eaters [compared to D2:ED] looked good, but the difficulty spike seemed insane. A Fire Sphere could only do about 5-10% damage, and unless they were at the far end of range, there was no way to avoid their seeking attacks. I started doing ok once by taking out the first couple nests and then circling around the area high up, but by the time I took out the first memory eater there were half a dozen more, making it impossible to survive.
After dying several times, I exited to the main menu rather than load a save, then decided to give it one more shot before I went to bed, so loaded the auto-save. The game seemingly changed the difficulty level (the setting remained on Nightmare) as this time the memory eaters were easy to defeat, attacks were relatively easy to avoid and the nests seemed slower spitting out opponents. I kept going when I was done, and the arena was also fairly easy.
The next day I reloaded the save at the Gremory chest, and double checked the difficulty setting, not wanting to make it through the Hall of Echoes only because the game decided to make it easier.
...
Next I returned to the battle tower and retrieved a couple pieces of dragon equipment from the chest, that had a shield skill bonus (and I put my 3 spare points into that skill), as well as the +40% defence bonus leggings (which I didn't end up needing). Earlier I removed all dragon equipment except for the Bone Dragon Cuirass (+40% hit points) and Bone Dragon Leggings (+40% offence) once I could max Firebreath, Fire Sphere, Dragon Spirit and Dragon Burst without the need for equipment spell bonuses.
I only had about 10 each of healing and rejuvenation potions, so farmed for enough ingredients to make a stack of each (while wearing my +3 Master Herbalist earrings, of course)...
Returning to the Hall of Echoes, the memory eaters were tough, but not nearly as much as in my first attempt (the Dragon Shield spell also helped a bit). The killers could be killed with 2 (or sometimes 3) Fire Spheres, while the generals needed 3-5, depending on how much they healed; attacks at mid range (that I could see coming) could usually be avoided. Two memory eaters at a time were not too dangerous, especially if they were close together. Three increased the difficulty, and 4 started to make things quite dangerous.
Unlike D2:ED, it was ineffective to circle around nests while using target lock (and having a flaky target lock in DKS didn't help either). I found it best if there was a memory eater in the distance when going after a nest to try and sit on the opposite side while using Firebreath (to hopefully block any seeking attacks), then hightail it out of there to avoid taking too much damage from the memory eater if it got close (fly, heal, then turn and use Fire Sphere).
The arena was fairly challenging. In the previous play session (when the game made things easy) I didn't even summon my creature, and think I only needed 1 potion. This time combat became much more hectic, with more and tougher summoned minions, which required more potions and strategic running away tactics in a couple places. My creature even died, which hasn't happened very much when it started off a fight at full health.
I'm assuming the difficulty level when I first entered the Hall of Echoes was a glitch, and the second was how it was meant to be. I don't see how it would be possible to take out all the nests with Fire Sphere doing so little damage to the memory eaters (the nests didn't seem to be particularly resistant to Firebreath). I also had a problem the first time I encountered the large group below the Grand Knight teleporter shrine in the fjords, where opponents barely took any damage, especially at long range (the next day I loaded the same save and tried again, and they took normal damage).