You can be in stealth when you jump. So jump all your party there before going further with Lae'zel. And if you go from the bottom of the tower, you can bring all your party without a problem.