The first star stone you got (crime scene or clinic) was triggered automatically, and opened the portal between the main area and the observatory (and teleported you there in the first place).
In the inner chamber, did you get a cutscene conversation triggered with an NPC there? Did you take the portal back to the main section of the End of Time at some point after that? (rather than leave using Rift Travel)
Does Zixzax have anything new to say? Is he standing near the weaver, or somewhere else?
How many inert stones do you have?