You can also fill up the amulet by consuming source orbs or virtue tokens if you have some of those in your possession.