Luckily I had a previous save point. Even though the ore does not trigger progress for the quest. You won't be able to finish the quest unless you have the ore. Kinda annoying.