You all means, "Everyone that what I said applies to." And that applies to everyone who thinks he is nothing more than a flamboyant idiot.
Again, I could be wrong, but I don't think he's an idiot at all. I think he's pretending to be an idiot to lull you into a false sense of security; writing him off as someone who is not a threat.
Yes, he threatens you on multiple occasions, but he does so mostly in jest... or rather masked in jests. That's my point. He's pretending to be weak and subservient and so forth, but I will certainly bet that he will become quite dark and sinister later in the game.
So, we'll see. I'm just not writing him off yet as only comic relief.
