i found that even if you were wearing the disguise, sometimes it doesn't work. If you engage one then depending on the proximty of other troops to you, they will turn against you to and then the rest. What I found to work the best was as stated before don't engage any troops but also play the area in sneak mode. Even in sneak mode you can still talk to everyone.
