Yes, you can swap musical instruments mid playthrough. Instruments appear as random loot AFAIK.

The level of detail in both sound and visuals, when casting bard spells is way beyond any other games I've played. I absolutely love it as well.