He approaches in a cut scene and says something to the effect that peasants should get out of the way of knights. Depending on your answers, he will either turn hostile or walk away. I answered neutral then defensive, but you can answer in a hostile manner right away; there is a pretty submissive answer to get him to leave. I meet him clearing the area south of the castle, west of the Rivertown Watch building.