For me it's as simple as that really: collars block source. Some spells can be used with the collars on. Those spells are not source material. End of the story. Going further is speculating and I don't give much care about it. I like it as it is.