I think it's better not to arrest him. He has some usefull things for sale: e.g. bottles of super poison. Very handy when you have the poison weapon skill.
Yes true, but his prices are high, and poison is not hard to come by. The exp points you get after turning him in make it worth your while.
