It depends on the mod you're trying to install.
Some come in a .pak format, which you have to place in:
Path\To\My Documents\Larian Studios\Divinity Original Sin\Mods\
And others are Public and/or Mods folders, which both go into:
C:\Program Files (x86)\Steam\SteamApps\common\Divinity - Original Sin\Data\
