As far as I understand it, weapon skills only add to damage. Hit chances are governed by other attributes. And so, for most characters, I tend to put next to nothing into that skill, since other factors seem to be more important.
Right now I even have a mage holding his own when attacked directly without having invested even one skill point into weapon or man at arms. Armor and weapon bonusses take care of that.