That's something you know because you habe played the game multiple times. The first time I (and I guess many others) just killed the fisherman. Which is not really a problem.
I did it on my first playthrough. I mean it's sorta obvious, you have a fight with puppets and puppet master, every time I get that kind of fight in an RPG I focus puppet master first.
