Yeah, just think about it and it makes sence. If you're gonna wear a really heavy metal plated armour for instance, it's gonna protect you well, but it's also gonna slow you down a bit.
The trick is to find armour that not only protects you well, but also doesn't affect your stats too negatively.