The die will pop up in certain situations where a check was made. As some examples, if you're in a situation where you walk by something that has a Perception check, it'll roll to see whether or not you've perceived/can interact with something, and they're also used for saves in resisting certain spells. It should just pop up, make a determination, and go away though. If it's always there, you're bugged.
