I have not run into this issue as a heavy Gale mm2 and scorching ray user.

However, Magic missile's trajectory does curve after the first cast. It could be very possible that you hit an object along the way, i've done it many times.