The total armour listed in the stats is just for your information. The game uses the armour class of where you get hit when determining how much damage is done, so not using a shield pretty much just means you will not be blocking as many attacks. You will also miss whatever bonuses or charm slots may be on a good shield.
As for swords, 2 handed swords do more damage, but are also slower. A faster weapon helps you attack first and more frequently, and will better interrupt your enemy's attacks (especially if you are also using the passive Stun skill). I used one handed swords for the beginning of the game, and then switched to the 2 handed sword dropped by the knight south of Stormfist castle (a couple levels after I got it) for the middle of the game. When I got the holy sword, I switched to it, and used the dragon shield I had been keeping at my home base.
