Not exactly logically equivalent but you are right, that line of argument does not make sense
I think it is a matter of principle for most people. Taking morals into the story makes it depend on a common moral standard that is just not there. Same rules for every species and race avoid such problems.