"Charmed, I'm sure" is not an insult; it's a form of cordial greeting towards someone whom you do not know well but wish to be polite to; it is formal, elegant and gracious in traditional form.

In some modern media it has come to be treated as innately sarcastic and mistrustful, but that is not its actual meaning or original use.