Making unlearning free would be unbalancing. It would make utility skills like identify/pickpocket/lockpick essentially useless, because you wouldn't need to commit points to them.
Making it low cost would not be unbalancing though. It would change the game mechanics, but in a good way that would make for interesting strategies. Imagine spending 2000 gold to shift 10 points into lockpick, unlock a few doors, then shift the points back to slashing damage.
Tom