Exactly. "I want more tension, where party members might leave if we don't do their thing, but I don't want them to nag us about doing their thing" seems to sum that up pretty well.
I can't tell if you're trying to point out a contradiction. If the characters
leave that solves the nagging problem neatly.