I assume it is just easier to prevent attacking NPCs than to keep track of which ones are essential and only allow minor or generic characters to be killed. For a 'feature' relatively few people would use, this would add a fair bit of work. If NPCs currently do not react to you taking their stuff right in front of them, then there probably isn't a system in place to allow them to react if you start slaughtering NPCs in full view, etc...
Ground opponents disappear in dragon form for balancing reasons. If you could 'toast goblins' then any open areas would be too easy as a dragon, or the game would have to have been balanced so that the human and dragon forms were equally powerful, and changing into a dragon was basically the equivalent to switching to a ranged weapon. Alternately, there would have to be creatures designed to be immune to dragon attacks (ie fire elementals or armour backed demons) but vulnerable to humans (other damage types, or attacks that could get below armour), or mobs would need to carry around some powerful ground to air weapons to use against dragons which would not be useful against humans.
