This has been an issue ever since the EA, and Larian ignored feedback on it. Since so much of the plot hinges on the artifact, it would make sense for the player to receive it immediately on the ship, by finding it. Unfortunately Larian has been weirdly insisting on Shadowheart carrying the box around. So the prism gains a teleporting mechanics (if you never recruit her, it will eventually show up in your inventory with zero explanation), and also apparently can work withing rather large range, as you can adventure for a quite soem time without it.
Imo the main story is badly written (there are other plot inconsistencies as you progress) and I don't think it can be fixed at this point, as it would require an extensive rewrite. So I just focus on the parts of the game I've found fun: exploration and adventuring with the companions I like.
