I have used it when someone is rattling on with complaints, questions or - especially - misconceptions. Whether or not they are using the phone. e.g. "Hold the phone, I never said you were an idiot I just said I was surprised that you invested in that ski hill in Florida."