After the anwsers no portal appears. If those guys are following you, then you did notting wrong. All is as it should be. If you leave the place, they dont follow anymore.

Joliekiller, paladin on a crusade against (almost) all evil.