The "Small Key" quest item is for a box on a shelf inside Farmer Jackson's house. Not only is it very easy to miss, the box can be opened with a high enough lockpicking skill, leaving the key stuck in your inventory if you picked it up after opening the box.
