You can give elementals items, the items disappear when the elemental despawns.

You can teach elementals skills via books; these skills are likewise gone when you next summon the elemental.

Trying to drag an elemental's weapon to your character's inventory crashes the game, regardless of whether the elemental has despawned.