That should just be the way games work as a standard. I've plagued plenty of games that do that so I would be dumbfounded if it worked any other way. Especially in a game like this where you are likely to be playing multiple games with different people running different mods.
This will be a huge downfall if you have to manually turn mods on and off. That alonecould kill modding and gm mode.
