Another way to save the cat is to talk to the undead dude below where the elves have their cave. Basically play hide and seek with the little boy in the cave and he will lead you to an undead dude petrified with a spear. He will give you instructions to unlock an underground passage from the main statue in fort joy. This leads underground into the prison. Go up the steps to a large door and in the hall way one of the paths will lead downstairs to where 2 guards are talking with a small child. Kill the guards ( There is a way to talk them down but that requires you to save the guy on the floor in the prison that is being interrogated) and then talk to the kid and he will tell you he can use the boat to take you out of the fort. Upon arriving on the beach just outside the walls the cat will be permanently rescued after taking about 3 steps away from the boat. If you have unfinished business just teleport back into the fort though I recommend maybe activating a checkpoint at the top of the hill from where the boat lands.
All of this will have you rescue the cat with only one fight and can take less than 15 minutes.
