I would be ok leaving someone behind if we had a party of six honestly, however commiting to 3 companions until the end of days feels too restrictive to me especially since we'll have at least 8 to choose from.
Seeing how I always drag my favourites along for the ride even in 100th playthrough (Minsc, Jaheira, Haer'Dalis, or Allistair, Morrigan and Zevran - yes this was a problem until mods) this could take a decade.