The more I play this game I can see where OP is coming from.
Well I have not had trouble with any encounter before the druid village.
Some fights almost verge on impossible (depending on your party), if your do not abuse:
1) Jump disengage
2) Preposition your party before battle starts
3) Hiding
4) Ground effects
5) Statuses that render enemy useless - entangled, unconscious etc.
6) Line of sight
I encountered the githyanki patrol on my second playthrough with a more squishy party - I didnt even get a turn because everybody was dead before I could act.
I was able to do it, by positioning properly before the fight, and using hiding effectively but as OP said I just barely scrapped by (three characters still died).
I think the end note here is that encounters should be doable without me feel like I exploited the game to get past them.