I believe Find Familiar is a ritual spell. Therefore if you cast it outside of combat, you don't actually use a spell slot. This is not explained in game but if you press T and hover over the ritual tag it will explain it in the tooltip. They really should show the ritual caster class feature for the classes that have them so this is more clear.
