I've noticed just how poor Swirl Attack is at hitting enemies. But I did find that it seems to poison them, regardless of your Accuracy! If you have the Poison skill, you can swirl at the start of combat and start weakening the whole group immediately.
Enjoy!