its by design
of course the design was untested because they released that as a patch i.e, people had to jump down holes one at a time so Larian 'fixed' that by adding a script to make the party follow unless they are unlinked
so yes the party falls to their death unless they have feather fall active and they jump in because Larian [in their wisdom] decided not to allow unlinking summons so [by script] your summon jumping in the hole is exactly the same as you jumping
p.s, yes they are well aware