But you can just as easily open the door using one of the books nearby.
Regarding the "room with the 4 baddies": you don't even need the key, since you are able to teleport/ jump and even cast (no projectile spells) through that door.