I noted that sometimes in Rivertown market it takes a couple of dozens of seconds to load the game, wheras it sometimes takes a minute in other areas.
EDIT: Ah, btw, if you want faster load times, go kill everyone. It helps, I tried ;0).