I got frustrated a number of times with that - Misty Step helped but shouldn't have been necessary. I know the exact encounter you're talking about, too, and 100% agree that there needs to be an easier way up to that specific rooftop.