Each suspect has both a smelly item you can pickpocket* and a smelly item you can steal from their home or workplace. You only need one for each suspect, and the whole "talk to the dog" is only one option for narrowing down the list.
In Evelyn's case, if you already had the dog smell her coat, you don't need to have him smell the scarf.
*You can also kill them to get this item, as you demonstrated, but except for Evelyn that approach will cause you other problems.