Don't forget that magic missile consists of 3 separate missiles. Casting it on a enemy with concentration should make them roll 3 concentration checks. Or, an enemy casting it on an unconscious party member makes them insta fail 3 death saving throws and die. So, while it doesn't do that much damage, it can still be pretty powerful.
Given how Larian has coded enemies to often focus on downed party members, I become so much more terrified when I face an enemy that can cast magic missile o.O
