Could it possibly be that being IN an active romance stops the other companions hitting on you? That would mean if you like romance, you start a romance arc in act 1 and don't get bothered by other requests, but if you don't like romance and turn everyone down, they keep hitting on you because you are supposedly single. Could that be the problem? This would explain why people to engage in romance don't see the problem.
