When Anders first gave the quest, I made a skill check (I think religion) after Anders said that this devil was going on a rampage with gnolls which said that "devil's can't collect souls through wanton slaughter" -- first clue that Anders isn't telling the truth.
Also, if you talk with the dead body in the Toll Building, (oddly named ''Dead refugee"), he tells you that they were able to kill the gnolls due to their "pact". I don't know if there was a skill check, but then there is a dialogue option like "you made a pact with a devil?", and the corpse says something to the effect that they made a pact with Zariel. After that, speaking with Anders gave me the option of "you're ruse is up .. I know that you're working for a devil." Also, after speaking to the dead body Lae'zal wanted me to speak with her and then warned me of getting involved in the affairs of devils.
Later on, when I confronted Anders accusing him of not being a Tyr worshipper he admits that he used to be a paladin of Tyr but is now fallen. Tyr didn't listen to him but Zariel did... And his pact with Zariel demands that he get Karlach's soul in exchange for his own. Zariel wants her because she was one of her former champions in the blood war but escaped (on the mindflayer ship).