They can't just remove combat sections because they're built into the map, essentially. If this was generating combat dynamically (randomized combat, with respawns, like you see in a lot of JRPGs), it would be doable. But less so in a narrative game that is built around a turn based combat mechanic. You can skip some fights, but you'll be weaker for when you really need it later, since there's no grinding.
If you don't mind going off on your own here and there, my recommendation is to unchain your character from the rest of the party (literally, using the game mechanics - not suggesting a real life separation!) - and wander around to do the NPC schmoozing/shopping/quest turn-ins, and let your partner handle the fights. On a lower difficulty level this is entirely doable.
You can sort of tell when they fights are inevitable via scouting the area (if you're in some subterranean cavern, there's likely going to be more stuff trying to eat your face) and using LEFT-SHIFT - that way you can see the red view-zones for monsters up ahead.
As for avoiding fights with NPCs, max persuasion and try to be polite? Helps if you're not playing Sebille or the Red Prince, too. Or (ahem) Lohse even in certain situations.