If you have a save before entering that cave you can reload and not take Bairdotr with you.

I don't know if there is a way to change the current party size value with the save game editor, or something.