I've noticed this as well... I'm not sure why it works for some wands and not others. I use a wand that does shadow damage and my int modifier works for it. Perhaps it's a bug...