Then you didn't answer right. You just have to remember what you DID, not what you ANSWERED in the situations the totem asked.

You don't have to answer what is 'right' to your characters.
It simply checks if you did what you said you did.

If you killed the guards, tell the totem the answer that looks like that.
If you stopped the thief, say the answer that you punished or stopped his hand.
If you helped Alfie, affirm that you did.

The first two questions are situations in Cyseal and the third is right after entering Luculla Forest, so you probably should have done them.