In a setting like FR you'd have to be careful making judgments based only on appearances.

Weak goblins could be warlocks, a stupid ogre could have exceptional intelligence, a massive spider could've been a spellcaster (hoping she gets cleric spells in release).

Otoh, being naive around feral goblins could end up you being in a pot. You'd have to pay attention to appearances too!

To survive, you'd really need to be part of a community that knows each other well, sticks together, and are suspicious of outsiders. And have fortified walls - or be hidden away. Thus the various factions.