I agree with you on most points, except I love the setpiece battles. I think there should be more some smaller encounters between them, but in general, very fun.
The issue is they need to be better thought out.
In my evil playthrough, when I raid the grove, Rath is still alive back at the inner sanctum entrance so I need to spend many rounds where I have to just use dash to get to him.
In the harpy battle, similar thing. Last harpy flew to other side of the map so spent 5-6 rounds just moving my team around.
There a point where the battle is over. When it's 4 to 1 in my favor and there no way I could lose, don't force me to spend like 15 minutes mindlessly having me move my people to finish off the last mob.
