If I recall correctly he'll instantly attack you if have Lohse with you.
If that's not what causing him to attack you in your current playthrough then I don't know what's causing him to be hostile, but I assume it's something you have done while exploring the island.
