Reason 1: too little charm slots Reason 2: amulet is a LOT better Reason 3: And you will get the sword of gods later on in the wastelands which is also A LOT better than the holy sword.
If you are in a sticky situation then the best tactic is to get out of it.