If you don't flirt with him and not raise his approval too much (read: don't take him along to bring Oliver back) he leaves you in peace - at least in camp. I don't know about any banter because I don't usually take him along.