As far as I know, you're going to have to hurt him.
Just say something, until you're forced to punch him out.