There are many options to kill a dragonrider, but most require some effort first.
1) get yourself a skillpoint in "deadly gift" an unleash killer scorpions on the dragon rider. 5 or 6 should be enough to kill him within a minute.
2) Try to find a weapon with frost damage, these blades are extremely powerful.
3) Attack the dragonrider with the spell elementary hail! (my favorite spell)
4) Curse the dragonrider first a few times and then use polymorph.
5) Use a shadow potion and then your swirl attack. Do not direct your attack directely on the rider but slightly beneath and you won´t become visible.
