I think Rory was referring to the mine itself, rather than the quarry tower. In any case there are a couple movable walls in the mine, but none in the tower that I found (or have seen mentioned).