The scaling is insane in this game and tactician only highlights that because its increase is % based. Act 1 is very open compared to act 2 (and future acts I'm told) You simply can't go around as you want to and expect to live. You level 9? Stay in Driftwood and do quests there. Lots of them just in town and you really need to get levels, and then gear to that level, before you explore north.
Just fought that scarecrow you mentioned today, party lvl 12, and the fight was quite easy. We had just gotten a few new epics so we were quite strong for our lvl. Our knight/necro could knockdown the adds with just a battle stomp, no setup needed, and our rouge 1 turn CC'd the boss. The scarecrows have very low armor, so exploit that and make a blood incarnate. At the time, my incarnetes charge hit for just under their full armor, so with a little help, easy to achieve knockdown there as well.
If you want the freedom of going up against opponents several levels higher than you, I think mods are needed. Otherwise the armor scaling is too brutal.
