In theory, you're loosing on some, possibly minor, stuff.
1) If you're main character is an origin one, there are some lines in dialogues you couldn't choose otherwise (i.e. as Ifan you can entirely avoid some confrontations, which wouldn't be possible otherwise).
2) Some solutions/options for origin character's quests are available only if you play as those characters, and you are able to unlock related achievements only when playing as them...............
What if I have these origin characters in my party with my custom main? Can't I just use the origin characters to interact with NPCs to achieve those origin tagged solutions/options?
For example, If my main is custom character and I have Ifan in my party, can't I just use Ifan to interact and use his dialogue lines to avoid the confrontation that you mentioned in point 1?
