I think it is supposed to change your eye color - I noticed that as well on my playthrough.
I also wondered about whether there was a way to reverse it. Maybe once you have access to the higher level restoration spells? It's a pretty major permanent debuff (even if you are asking for it). It would be nice if the hag's eye amulet provided relief, or if it was mostly bad but it gave you a +1 on intimidation checks or something.
