The problem I have is that most fight currently for me are non-challenging, but they still take several minutes of waiting for animations to finish playing to resolve.

My suggestions for speeding combat up:
Take Glass Cannon on everyone
Set Difficulty to Hard

You now deal much more damage per turn, and have much less HP.