I tried stealing the stone, but it did not appear in my inventory- gave me some bonus xp though.
I ended up reloading just in case that this breaks something later.