If you keep the sword, the ghost will eventually kill the inhabitants of the Dark Forest (except Bronthion).

I'd leave the sword in there and return when you have solved all Dark Forest quests. Your elven love will still die - but you have no XP losses.

Political correct: Draw the sword - defeat ghost once, drag&drop sword back into the stone when it glows.
Kiya