I've heard a number of complaints about how items are added to the skillbar. I made a quick mod for you guys that prevents Objects and Potions from being automatically added to your skillbar.
I can only hope that Larian Studios makes this a clientside feature that can be toggled on or off via the gameplay options. As it stands that's not an option so here's the link: Tidy Skillbars by Rykari
If there are any problems let me know here or on nm.
Works on multiplayer:
Host must have it installed for it to work. Nobody else needs it installed.
You don't need to uninstall the mod to play with other people who don't have the mod, just know that it wont work unless you're hosting
Much love xo