if you were verbally cutting someone down to size (and i expect your grandfather wasn't jack the ripper, or the man behind the "texas chain saw .." )-- it would have been cuffed or boxed your ears-- though it wasn't unknown for a mother to suggest you should listen to her, and to make her point--she would grab you by your ear lobe.. Never happened to me, but i have seen it done...

We keeped our ears "pinned" for news...
and one cousin with "jug handle ears" was once found in his early teen with scotch tape on his ears, trying to make them lie a bit closer to his head.. he would have been quite happy to pin back his ears...