You can go back after broken valley is poisoned. In fact scenario inside the quarry building is design for you to come back later, as those black ring inside are at around lvl27,28. That is why when you first approached the quarry, "Talana" had warned you to come back later for the black ring elites, which is referring to those inside the building.
