I just tried this and didn't have a problem learning Staff of Magus (automatically), but unlearned 3 of 5 Huntsman skills when switching back to a crossbow.