[color:"orange"]Whenever I come close, only the dialogue box opens, nothing else.[/color]
This is the original dialog, where he tells you that you can have the key? Picking up the key should trigger another conversation, after which he turns hostile.
A couple other people ran into a different missing quest item, which also should have been on the ground. The workaround for that should also work here (see below);
3rd. Crystal Fragment help (minor spoilers)