You do actually. Constitution is used for hp and for concentration checks. Even as a sorcerer or wizard, you do need some buffer for your health. You won't be doing any good to your party if you drop unconscious in round 1.
P.S. I assume our sith friend meant that he dumps intelligence when that stat is not a primary stat for the class.
Don't think I've ever had constitution above 12. Even for caster classes it's usually a third tier stat.
