Just to mention, the line in the game isn't "I met a man who wasn't there."
The line is, "I met a man who wasn't a man."
Here is the quote from the mason's log book:
- Sick of standing idle while Justiciars gain power in our humble town. What will become of us if we allow it? I met a man who wasn't a man. Touched by a devil. Or maybe worse. But he offered me something I couldn't refuse - help.
This is, I believe, a reference to Raphael. The stonemason made a pact with Raphael for help. That's why Ketheric was defeated a hundred years ago. That's why Grymforge became what it is. Yurgir and his army of merrigons invaded. Yurgir had a quest to get rid of every last justiciar. Of course, Raphael also made a deal with the last justiciar, splitting him into numerous rats so that Yurgir would be forever stuck trying to finish his task.