No, they are systems. They're just heavily reliant upon writing. All the options and reactions have to be written and the writing has to enable the multiple options to adhere to the overall narrative. Unlike say, combat where its an entirely closed system where it doesn't matter to anything else other than the combat system what the combat system is doing.
I don't think they are. Aren't dialogues closer to scripting than a system? Systems suggest to me rules and verbs on account of players. There are no "ask", "question" "flirt" options. Just manually connected nodes with no systemic meaning by themselves, used to deliver mostly uninteractive content (text or cutscenes), and in the best case scenario feeding back and forth to game proper systems (reputations/character build)
So what have changed is amount, quality and presentation of written text, complexity of scripting and systems the scripting connects to.
To really shake up conversations one would need to abstract conversations and adapt them into a gameplay system, but I am doubtful if it is an objective woth pursuing. Games struggle with far simpler concepts (like crime) and I doubt one can come up with consistant ruleset for "developing a relationship with a fellow sentient being".