No need to punch anyone even. Cast Friends cantrip on the ones you want out.

Each cast nets -10 approval. They give two warnings and permanently leave at around -40.