I send Keys to the camp-chest after use.
Not like I will ever clean that box up in playthroughs I will never hang with for a longer time anyway.
But indeed, once a Key has fulfilled its purpose, its only in the way.