How about no armour at all? Heck, you've already made a game without any normal clothes - I think a game without armour, just with standard clothes, could both be more challenging and more interesting. It wouldn't be the first RPG without armour, though: Jade Empire also didn't have any armour, just amulets.
And looting armour-clad enemy corpses wouldn't necessarily help ... size does matter, after all, and it isn't very logical that you can put on a slain goblin's plate mail as a human.
If there is armour in the game, however, NPCs who can change armour size might be an interesting addition. You might need two goblin-sized sets of armour to create one human-sized set, or there might even be a quest to find some rare materials if you want to re-size a valuable set without making it lose its magical properties.
