The issue is you're not necessarily playing a reasonable person. Through your direct actions, you can get those kids killed via the goblin route. It just seems jarring that a psychotic villain won't lift a finger against the kids directly after they've stolen from them.
+1