In case you didn't read my edit, it turns out I was blind and missing the church key in plain sight.
In the Blacksmith's basement, there's a pressure plate you can step on to trigger some fire traps. There's also a lever you can use to shut off the traps, and a nearby oil barrel. That gave me an idea: I placed the oil barrel on the pressure plate. That gave the click indicating it was triggered. Then I went to shut off the traps, and I figured that since the plate was already depressed, stepping on it again shouldn't do anything, and the traps should stay off.
Unfortunately, that is not the case, and even though the plate was held down and the traps were off, stepping on the plate again just started them up. I'd really like it if that worked. It's one of those clever thinking things.