I was able to avoid it the first time playing. yes they saw me and killed me before I knew they were there, but after a reload I just snuck around them and avoided that section all together. nothing back there beyond a chest I think. Can walk down the other path and jump across to get to the mushroom people and not have to fight the minotaurs
It's possible only if you come from the goblin fort and only if you know they are there. Hardly optional imo. Again, I don't mind it not being optional provided we'll be allowed to have an extra level or two when fighting them (which I believe we'll have)