My only concern is possibly depriving other party members of their essential skills (Since you can't remove or copy a skill you learn as it devours the book)

Especially in a quasi-competitive game where making sure you don't get certain skills might be beneficial to them.