.... Just to be clear, this was only the case with Fergus when he was still in the room and I couldn't see him. Didn't notice it with guards who were holding keys.
Not sure if its related... but when the doors are locked, sometimes from a certain spot you can see the enemies inside the next room perfectly.