While I would benefit greatly from this change, I think we also need to be mindful of the impact it may have on performance.
The current party management system is probably also a way to optimize the way in which the game accesses follower inventory in the memory.
Suddenly retrieving the entire party's inventory on-demand might not be as smooth as the current implementation.
Just wanna make sure we got our expectations straight before such a change would go live and everyone's like "inventory loading slower after update".