He isn't a skeleton.
Since D&D has it's monster lore, he seems to be a lich of some kind, some kind of immortal creature with body issues as it's body is dead. The question is whatever he is some kind of divine avatar or other super divine servant.
Making him leave is on the same level as not wanting "the box", you are not going to get a choice in the matter he is part of the fabric of the story, so the choice is do not play BG3 and avoid Mindflayers, Withers and "the box" or accept it as part of the game.
