You get conversations with Astarte or another character in each portal that you open. Avoiding talking to them (where it is not automatically triggered) will just delay that room being completely open, so no new portals will be activated.