I like the idea of a flavor feat and a system I saw once, not sure if it is in 5E, is to take disadvantages as well. In game for example Shadowheart freezes up around wolves.