... Simon Singh's The Code Book is a brilliant cyphers-for-amateurs work (and there's a large-ish prize at the end, if it hasn't been claimed yet).

1. Letter frequency: Usually, I believe worked out from various corpuses, of a million words or more each. Simon Singh gives various frequency details from different corpuses. In any large enough body of words in English we find a surprisingly similar set of frequencies. Etaoin shrdlu may not be an absolute representation of relative frequencies, but it is a very useful one for a codebreaker (yes I know it's technically called a cypher, but if Simon Singh can call it code, I can too).

2. Letter frequency is not used in any absolute sense when codebreaking (ok, ok, deciphering). See, for instance, the classic story - Sherlock Holmes' "Adventure of the dancing men", to see how letter frequency combines with intuition and good ol' fashioned guesswork, to make decipherment possible. All codebreakers rely upon getting their hands on a reasonably susbtantial portion of text, and for whatever reason, I believe that type of use (telegraphic, legal, medical etc) does not skew the frequency distribution too much.

3. The technique was developed by (who else?) our old friends the Arabs (Saracens?), and the word cypher (for another cross-thread) comes from the Arabic (sifr?). In the tenth or eleventh centuries, the agents of the Caliph and others, determined to crack letter-substitution codes, slowly built up their knowledge of letter frequencies in Arabic. Whether this technique was slowly disseminated through Europe, or re-invented later, I don't know, but it has stood codebreakers in good stead for centuries. As Simon Singh points out - Mary Queen of Scots lost her head because of the efforts of a codebreaker using letter frequency tables to help him decipher her secret messages plotting against Lizzie 1.

cheer

the sunshine warrior