and getting five of them would indicate that you weren't able to graduate in the normative four years

Maybe not. Calling someone a "three letter man" is a high compliment, meaning he is a so good a college athlete as to letter in three separate seasonal sports: a fall sport, a winter sport, and a spring sport.

Is the term five letter man is use? I can't recall it, but am unsure.