Yeah, if you only acquire 4 companions total, it's... an option.
Problems: Tavs cannot be removed from the party. ! Scenes now only between Tavs. So, e.g. instead of Gale talking about the hag outcome with you, the two Tavs will now talk among each other (you're essentially playing two multiplayer characters at the same time). The two Tavs also have different affection meters to companions.
Because of this, it might be better to wait slightly longer and grab a mercenary . Larian apparently said, however, there's "1-2 more companions". Also, Minthara is properly recruitable on the evil side. There's hope some of the "1-2" companions are more likeable, but they're also probably later recruits.
As I also mentioned, Wyll has been majorly reworked. He even has a new voice actor. He might be tolerable enough for a while. However, you'll need to smooth out some things with Karlach in that case. If you're lucky, this is an early defining event and it sets him on the good/less obnoxious path.