I have also since this happen from within dungeons. So I think the pyramids doesn't work across maps in general.
They used to work in all cases back in Divine Divinity, so I suspect it is a bug or maybe due to some technical limitation...
I suspect technical - part of that cleaning up that happens when you change maps maybe?
