Accuracy is mostly determined by agility, and partly by survival. Speed influences your attack and hit recovery rates.

The one handed damage skills are completely separate from one handed with shield skills, so make sure you are boosting the appropriate skill.