It depends on the builds and how you use it. For me the biggest gripe with the game is not the ap system or the armor, but the easy way to get a combi of skills from several schools which is not healthy.
After some rough testing (I did not play the first game) I got a bit used to the combat. I play relatively straight builds, with a restriction of schools per char, for example I have Adrenaline only on my rogue and Skin Graft only on my Necro/Poly and only one Teleport in the group.
That said, I don't find the first four levels so hard and then the rest boring. There are easy fights and difficult fights on all levels. I lost a fight against the frogs, the magical undead in the tomb, one against Kniles and two of five against Radeka. I mostly had the feeling I had to work a bit for wins. More than a bit in the Voiddweller fights for example.
I like the ap system as it is. It is restrictive and urges you to use some tactics. I don't see much sense to implement an increase of ap when at the same time stronger skills use more ap. I don't like the source points, actually I never ever used a source skill in combat.
