You should be able to edit your hotbar to drag the improvised weapon off it entirely or move it to a different spot where it’s not visible on the combined view.
I must admit I’ve hardly ever used improvised weapons, perhaps because I don’t tend to run with strength-based characters, but I don’t mind having the icon there to remind me it’s an option. I’ve always taken it that it’s more for flavour, fun and roleplay than because it’s going to be the most effective way of doing damage. But perhaps there are builds that can make better use of it than I’ve considered!
