It was a bug apparently, I came back to him after killing the 5 contract (the dragon voice told me it was time to go to Maxos temple) and I defeated him without problem.

Thanks for your help anyway ;-)