Yea, respecing breaks the stat boosts. Pretty sure this happens because the game goes to a different instance when you enter the respec mirror and when the game makes the stat changes, they were stored as instance variables instead of setting them in stone. Quicksave/quickload usually fixes it, but you would have to do this every time you respec.
