You mean that you don't get anything for your summon reducing their vit?
So a point in necro would be pointless for a summoner?
They would still benefit with Dimension Bolt, weapon attacks, and any cross classing attack.
But yes their mainstay definitely doesn't benefit (nor does Necromancy really mesh all that much with the skillset anyhow)