It happened to be these totem imps (forget their proper names) that I basically had to target and kill super quick, lest they lob a fireball at me for 1300+ damage. The fireball would kill me even if it passed nearby - very frustrating.