I did the same thing but fortunately had some bombs with me which took care of it. Seems like a a bug that nothing else can damage the rubble.