The only balanced option is removing it from the game. How is it balanced that I only need to invest 1 point into Polymorph to get a skill that effectively DOUBLES my damage output?

Take any build you want. Add Skin Graft to it. Your build is twice as good.