I support this for others, but until party UI isn't fixed I wouldn't want 6 characters, 4 is too much already. Party UI makes me to solo most of the time anyway (just before a fight starts I move the rest). Seems like off-topic, but party without proper UI is a nightmare.
Default party can be any size. Game can be balanced for any, if stays 4 is no problem. And then allow as optional party 5 & 6. Feels like a different game with a proper party composition.
Also adds another layer to difficulty options: easy with party of 4, hard with party of 6 and so on.