I think magic has always been a non-hostile option: I forget if he initiates the conversation or if I do, but he always gives me the sparkler card to use with the gamblers as long as I don't antagonise him. Though attempting to give him a potion seems to ensure he gets in a bit of a strop.
