yes. it's possible. they are level 8 though, so you'd better skip the fight and do everything in the fort before fighting them.
there are few ways to skips them, so I guess you can find them without spoilers from me. do mind though that if you escape the fort they'll disappear so you have to do it before you leave the fort joy.
as for affecting last battle...iirc there will be additional dialogue line
