I personally think 4 is enough, but I wouldn't mind a party 5. 6 is too much, make less sense in a dungeon, because it's too crowded.
DnD sessions are usually between 3-6 people, on average 4-5, having too many players is always a hassle.