Yes, I would also rather choose what I put on there myself. This sounds like the case for an "[x] Automatically add usable items to quickbar" option under Options > Gameplay. On by default, but you can turn it off.