The book says you need to drop the mirror first, but yes, it does work both ways.
It also says you can duplicate "certain magical items". I remember once holding on to the mirror until I had the level up jar. That was some years ago and don't recall if it worked or not. (Probably didn't because you still only have the one key.) But perhaps there are other things that can be duplicated, like charms. Perhaps someone should test that!