Wordsmith.org: the magic of words

Wordsmith Talk

About Us | What's New | Search | Site Map | Contact Us  

Previous Thread
Next Thread
Print Thread
Page 1 of 4 1 2 3 4
#11494 11/29/00 09:52 AM
Joined: Mar 2000
Posts: 1,004
shanks Offline OP
old hand
OP Offline
old hand
Joined: Mar 2000
Posts: 1,004
Ann's postings on typesetting set me off on this. It's more a challenge than anything else, but fun anyway. What is the significance of the title of my post - both to the world of typesetting, and to that of code-breaking!

cheer

the sunshine warrior


#11495 11/29/00 03:57 PM
Joined: Apr 2000
Posts: 10,542
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Apr 2000
Posts: 10,542
"With the idea of speeding up..." research, here is Michael Quinion:
http://www.quinion.com/words/weirdwords/ww-eta1.htm

(but he only hints at the code-breaking part...)


#11496 11/29/00 05:15 PM
Joined: Nov 2000
Posts: 1,289
B
veteran
Offline
veteran
B
Joined: Nov 2000
Posts: 1,289
It's the frequency distribution of the letters in English words. See Sherlock Holmes in "The Dancing Men". It may be noted that there are other lists with different arrangements.


#11497 11/29/00 06:09 PM
Joined: Sep 2000
Posts: 2,788
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Sep 2000
Posts: 2,788

A sophistication of the letter frequency method for cracking a simple code is provided by a knowlede of which digraphs and trigraphs appear most frequently in English, e.g. TH, HE, IN, ER, RE, AN, ON and IN; THE, AND, TIO, ATI, FOR, THA, TER and RES.



#11498 11/29/00 07:36 PM
Joined: Nov 2000
Posts: 3,439
W
wow Offline
Carpal Tunnel
Offline
Carpal Tunnel
W
Joined: Nov 2000
Posts: 3,439
Ok, OK, I got note from shanks re this subject and sent him a private note declining to be the one to introduce it. He has more courage than I do so he brought it up.
L'audace, l'audace, toujours l'audace is this Board's motto!
I have no idea about the cipher aspect, being hopeless at codes. That is all new and I will try to follow the answers on that. Good luck to me.
I ran into the phrase when my Dad first showed me a Linotype machine. Hey, I'm talking me as a toddler in the 1930s. Yes, I am an oldie.
The letters on the keyboard of a Linotype are not set up the same as a typewriter and the men who did typesetting were incredibly fast on their machines but could not transfer the skill, at speed, to a typewriter.
The Linotype makes lines of type from pigs of lead fed into the machine and heated to melt and reformed into letters and lines, the lines become banks of type that are locked into a form which later becomes part of a page. If a typesetter made a mistake he would make an X on the keyboard (starting in upper left) and the type would set as ETAOIN SHRDLU. - (My favorite swear word) - Typesetters' eyes could pick out the X'd line at a glance and remove both it and the line before it where the mistake was made. So there you go, that's my explanation.
As an aside : when visitors came through the typesetting area the operators would often set the persons name in type for them which the typesetter handed to the visitor. The men's hands were toughened to the heat but visitors got a gift of HOT type and the typesetter got a chuckle and a "gotcha" for slowing the work.
Linotype operators (typesetters) belong to one of the oldest unions. Newspapers were among the first businesses to be "automated." That's another thread for a different board.
If any of you have a newspaper near you that still uses one of the great HOE presses, ask for a tour. You'll learn a bit about how type is made, how the technique was apllied to computers, and you'll have a great time.
That's it. I am off the printing, typesetting thread for good. Why did I ever start with it? Not another word. No explanations for type lice or buckets of steam or left-handed quion keys, or chocolate kerns, or a 10 point minion, or a woolen offset blanket. And if shanks wants to go off on horseback and also intoduce the keyboards used by signalmen in WWII well, don't expect a word from me. (I heard that!)
Blessings on you all. wow


#11499 11/29/00 07:41 PM
Joined: Nov 2000
Posts: 3,439
W
wow Offline
Carpal Tunnel
Offline
Carpal Tunnel
W
Joined: Nov 2000
Posts: 3,439
This is really my last word...I read the link and I admit my mistake....it was not an X, rather the top lines of the keyboard. I was a five-year-old for heaven's sake!
wow


#11500 11/30/00 09:04 AM
Joined: Mar 2000
Posts: 1,004
shanks Offline OP
old hand
OP Offline
old hand
Joined: Mar 2000
Posts: 1,004
And is probably the only way to crack codes of the Playfair type - with doublets being transposed?


#11501 11/30/00 09:08 AM
Joined: Mar 2000
Posts: 1,004
shanks Offline OP
old hand
OP Offline
old hand
Joined: Mar 2000
Posts: 1,004
Thanks for that Ann.

The point being (just stating the obvious here) that they were on the top for the logical reason that they are the most frequently used letters - as most of the code-breakers below have appreciated. If we hadn't the historical accident of qwerty keyboards, we might well be using etaoin keyboards these days...


#11502 11/30/00 01:32 PM
Joined: Oct 2000
Posts: 1,346
F
veteran
Offline
veteran
F
Joined: Oct 2000
Posts: 1,346
codes of the Playfair type

Sounds like a merger between Playboy and Mayfair!

Reference or explanation please, shanks.



#11503 11/30/00 02:17 PM
Joined: Nov 2000
Posts: 3,439
W
wow Offline
Carpal Tunnel
Offline
Carpal Tunnel
W
Joined: Nov 2000
Posts: 3,439
May I recommend a book to those who love ciphers/codes/good story ? "From SIlk To Cyanide" by Marks. Published within the last year in US. As noted I am hopeless with codes but this is also a whacking good tale, and true. The writer is the son of the owner-founder of Marks & Co of London, a bookstore that is featured in the delightful tale "34 Charing Cross Road." The story is set in England in WWII years where he was in the business of helping spies. Anyone who has done some reading on WWII will recognize the names of many spies who became famous after the war when their work and sacrifice were finally recognized. Happy reading, wow


#11504 11/30/00 03:11 PM
Joined: Nov 2000
Posts: 1,289
B
veteran
Offline
veteran
B
Joined: Nov 2000
Posts: 1,289
In reply to:

codes of the Playfair type



A Playfair cipher (it's not really a code)substitutes letters in pairs from a simple grid. Dorothy Sayers, who was fond of such problems, gave a detailed description of the cipher and how to crack it when she has Lord Peter and Harriet discover and solve one in Have His Carcass.


#11505 11/30/00 08:07 PM
Joined: Jul 2000
Posts: 3,467
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Jul 2000
Posts: 3,467
WOW:

What memories you brought back. My parents were both journalists, one a reporter and the other an editor. My mother was a police and courts reporter for the Alexandria, Virginia Gazette, which boasted that it was the oldest continuous daily newspaper in the US. Now defunct, of course, much to my dismay.

I still have my line of type with my name on it that I received on a visit to the press room not quite 50 years ago. And I definitely remember the strange keyboards and the even stranger men who worked at them. They all smoked unfiltered Camels which they lit by touching the tip to the bucket of hot lead which fed the machine its raw materials.

In fact, they still set some type by hand for display ads when I first started hanging around. I too wish I had known now much the cases would be worth, because there were stacks of them, many still containing their fonts, gathering dust in a shed behind the building.

Those were glorious times, and I suspect the world is poorer for the changes that have come to the newspaper world.



TEd
#11506 12/01/00 06:42 AM
Joined: Oct 2000
Posts: 197
X
member
Offline
member
X
Joined: Oct 2000
Posts: 197
historical accident of qwerty keyboards

Not an accident at all. They made the qwerty so that the common patterns of keystrokes would be far apart so that the typewriter wouldn't get tangled up as easily.

it's a conspiracy, i tell you

p.s. why does AEnigma interpret qwerty as rabbit?

It's time for the human race to enter the solar system.
--Governor George W. Bush Jr.

#11507 12/01/00 08:19 AM
Joined: Mar 2000
Posts: 1,981
J
jmh Offline
Pooh-Bah
Offline
Pooh-Bah
J
Joined: Mar 2000
Posts: 1,981
>They all smoked unfiltered Camels which they lit by touching the tip to the bucket of hot lead

Yes, those Health & Safety people spoil all the fun these days, don't they? I remember the days when chemistry lessons had just the right number of explosions per lesson to keep everyone awake, they stopped those too.


#11508 12/01/00 09:38 AM
Joined: Mar 2000
Posts: 1,004
shanks Offline OP
old hand
OP Offline
old hand
Joined: Mar 2000
Posts: 1,004
historical accident of qwerty keyboards

Not an accident at all. They made the qwerty so that the common patterns of keystrokes would be far apart so that the typewriter wouldn't get tangled up as easily.


True enough, xara, but one presumes that there were a number of ways in which this could have been done, so the particular arrangement we have is, to that extent, a matter of chance. More importantly, however, there was the opportunity to change to a more rational scheme (a simple alphabetic one, if I remember rightly) that we missed because the winner of a touch-typing competition around 120 years ago happened to be a man who used a qwerty keyboard. As a result, qwerty became established (for speed! not for slowness!), and is now ubiquitous (except, as we can see, amongst linotype operators).




#11509 12/01/00 10:28 AM
Joined: Oct 2000
Posts: 1,346
F
veteran
Offline
veteran
F
Joined: Oct 2000
Posts: 1,346
they are the most frequently used letters - as most of the code-breakers below have appreciated


Now, hold it here a second, folks. It just occurs to me that I've always blithely accepted statements about "the most frequently used letters" and, very closely related, "the most commonly used words" in the English language.

How are these "facts" known??

OK, in a particular context for a certain period of time you can make a broad-sweep judgement. Also it's a lot easier these days to produce accurate mathematical counts for a number of documents, and then extrapolate from there.

But but but.. like all statistics, we don't have any absolutes. In many contexts, for instance, articles will be omitted, words abbreviated and slang introduced. Surely this makes a significant difference to word and letter counts. And another thing - language evolves over time. New words and phrases appear, and old ones fade away. Sometimes (as we know) this happens very quickly.

Codebreakers would certainly need to take account of such factors, as top secret messages are only going to say what has to be said, in as condensed a form as possible. Context is absolutely crucial, as that will dictate the most frequent phrases/words/abbreviations.

I'd therefore like to ask:
Who came out with these frequency figures, how were they determined, and when?

Nice simple question!






#11510 12/01/00 11:05 AM
Joined: Mar 2000
Posts: 1,981
J
jmh Offline
Pooh-Bah
Offline
Pooh-Bah
J
Joined: Mar 2000
Posts: 1,981
So would you go with Dvorak then?


#11511 12/01/00 12:11 PM
Joined: Jul 2000
Posts: 3,467
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Jul 2000
Posts: 3,467
No, thanks. I prefer Sibelius myself :)



TEd
#11512 12/01/00 01:36 PM
Joined: Oct 2000
Posts: 1,346
F
veteran
Offline
veteran
F
Joined: Oct 2000
Posts: 1,346
So would you go with Dvorak then?

Nah, I'll just go with the flow, Jo!

Zen Fish
(Zen bones)
(Zen - dry bones..)




Joined: Mar 2000
Posts: 1,004
shanks Offline OP
old hand
OP Offline
old hand
Joined: Mar 2000
Posts: 1,004
... Simon Singh's The Code Book is a brilliant cyphers-for-amateurs work (and there's a large-ish prize at the end, if it hasn't been claimed yet).

1. Letter frequency: Usually, I believe worked out from various corpuses, of a million words or more each. Simon Singh gives various frequency details from different corpuses. In any large enough body of words in English we find a surprisingly similar set of frequencies. Etaoin shrdlu may not be an absolute representation of relative frequencies, but it is a very useful one for a codebreaker (yes I know it's technically called a cypher, but if Simon Singh can call it code, I can too).

2. Letter frequency is not used in any absolute sense when codebreaking (ok, ok, deciphering). See, for instance, the classic story - Sherlock Holmes' "Adventure of the dancing men", to see how letter frequency combines with intuition and good ol' fashioned guesswork, to make decipherment possible. All codebreakers rely upon getting their hands on a reasonably susbtantial portion of text, and for whatever reason, I believe that type of use (telegraphic, legal, medical etc) does not skew the frequency distribution too much.

3. The technique was developed by (who else?) our old friends the Arabs (Saracens?), and the word cypher (for another cross-thread) comes from the Arabic (sifr?). In the tenth or eleventh centuries, the agents of the Caliph and others, determined to crack letter-substitution codes, slowly built up their knowledge of letter frequencies in Arabic. Whether this technique was slowly disseminated through Europe, or re-invented later, I don't know, but it has stood codebreakers in good stead for centuries. As Simon Singh points out - Mary Queen of Scots lost her head because of the efforts of a codebreaker using letter frequency tables to help him decipher her secret messages plotting against Lizzie 1.

cheer

the sunshine warrior


#11514 12/01/00 02:29 PM
Joined: Nov 2000
Posts: 3,439
W
wow Offline
Carpal Tunnel
Offline
Carpal Tunnel
W
Joined: Nov 2000
Posts: 3,439
Dear FOB, Indeed, transpositions and bad spelling often caused problems for code-breakers especially with the spies sending messages under the duress of combat conditions encountered in WWII. Again, I plug "From Silk To Cyanide" where this discussed in depth with personal experience by author Marks (who was a mere stripling at the time.). ALERT: this is not yet another book about the Enigma machine at Bletchley (sp?) but a different, English, operation. The code-breakers Marks worked with were all women, for one thing! 'Nuff said.
wow


#11515 12/01/00 02:48 PM
Joined: Apr 2000
Posts: 10,542
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Apr 2000
Posts: 10,542
>The code-breakers Marks worked with were all women, for one thing!

...as were many at Bletchley. http://www.secretsofwar.com/html/bletchley_park.html


#11516 12/01/00 03:53 PM
Joined: Apr 2000
Posts: 10,542
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Apr 2000
Posts: 10,542
coincidentally, today's M-W word-of-the-day is cryptography. is somebody spying on us?!


#11517 12/01/00 07:38 PM
Joined: Jul 2000
Posts: 1,094
J
old hand
Offline
old hand
J
Joined: Jul 2000
Posts: 1,094
p.s. why does AEnigma interpret qwerty as rabbit?

Apparently, AEnigma has an alphabetical list of all the words that it thinks are acceptable. If it finds a word that it doesn't recognize it just goes to the next word on the list. (This board isn't written in the most complex of computer languages, so the spell check was obviously kept simple.)


#11518 12/01/00 10:20 PM
Joined: Mar 2000
Posts: 1,981
J
jmh Offline
Pooh-Bah
Offline
Pooh-Bah
J
Joined: Mar 2000
Posts: 1,981
>to the next word on the list

In which case, the first "r" word must be rabbit. Or maybe Aenigma is recognising that it is the first day of the month.

Assuming the first version is true - Can anyone, without resorting to a dictionary, on-line or otherwise help Aenigma out with the missing words?


#11519 12/01/00 10:39 PM
Joined: Apr 2000
Posts: 10,542
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Apr 2000
Posts: 10,542
if you did resort to something like the OED there are probably lots, but the only significant ones I can think of are Ra and rabbi (plus rabbinical etc.)


#11520 12/01/00 10:43 PM
Joined: Jul 2000
Posts: 1,094
J
old hand
Offline
old hand
J
Joined: Jul 2000
Posts: 1,094
Assuming the first version is true - Can anyone, without resorting to a dictionary, on-line or otherwise help Aenigma out with the missing words?

There is no way for us as board users to add more words to Aenigma's lexicon. The only people that could do that are those who wrote the code for the board's amazing spell check feature.


#11521 12/01/00 10:45 PM
Joined: Mar 2000
Posts: 1,981
J
jmh Offline
Pooh-Bah
Offline
Pooh-Bah
J
Joined: Mar 2000
Posts: 1,981
>There is no way for us as board users to add more words to Aenigma's lexicon

I wasn't thinking of adding them, just wondering what Aenigma was missing out on. Sounds like rabbi postings are a bit of a no no.


#11522 12/02/00 01:50 PM
Joined: Nov 2000
Posts: 3,146
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Nov 2000
Posts: 3,146
Actually, etaoin shrdlu is simply the order of the letters on the first two vertical rows on the left hand side of the linotype keyboard. The reason that it became so well known is also simple - bad proofreading. Often, if a linotype operator made a typing mistake, rather than being bothered to fix it s/he would simply run a finger down the first two rows of the keyboard to justify the line, with a couple of spacebands to take up the slack, then cast the line anyway. If the operator didn't diss it or the proofreaders didn't spot it, it wound up in print.

So yes, it was a cipher - but for "I screwed up", not anything more sinister!



The idiot also known as Capfka ...
#11523 12/02/00 09:01 PM
Joined: Oct 2000
Posts: 1,346
F
veteran
Offline
veteran
F
Joined: Oct 2000
Posts: 1,346
is somebody spying on us?!

"Just because you're paranaoid doesn't mean they aren't out to get you."






#11524 12/04/00 12:29 PM
Joined: Mar 2000
Posts: 1,004
shanks Offline OP
old hand
OP Offline
old hand
Joined: Mar 2000
Posts: 1,004
Actually, etaoin shrdlu is simply the order of the letters on the first two vertical rows on the left hand side of the linotype keyboard. [snip] So yes, it was a cipher - but for "I screwed up", not anything more sinister!

My point was that the reason for thie being in that position was that they tend to be (in that order) the commonest letters in the English language. This knowledge (of letter frequency) is invaluable when attempting to decipher letter-substitution methods of coding...


Page 1 of 4 1 2 3 4

Moderated by  Jackie 

Link Copied to Clipboard
Forum Statistics
Forums16
Topics13,912
Posts229,283
Members9,179
Most Online3,341
Dec 9th, 2011
Newest Members
TRIALNERRA, befuddledmind, KILL_YOUR_SUV, Heather_Turey, Standy
9,179 Registered Users
Who's Online Now
0 members (), 676 guests, and 4 robots.
Key: Admin, Global Mod, Mod
Top Posters(30 Days)
Top Posters
wwh 13,858
Faldage 13,803
Jackie 11,613
tsuwm 10,542
wofahulicodoc 10,510
LukeJavan8 9,916
AnnaStrophic 6,511
Wordwind 6,296
of troy 5,400
Disclaimer: Wordsmith.org is not responsible for views expressed on this site. Use of this forum is at your own risk and liability - you agree to hold Wordsmith.org and its associates harmless as a condition of using it.

Home | Today's Word | Yesterday's Word | Subscribe | FAQ | Archives | Search | Feedback
Wordsmith Talk | Wordsmith Chat

© 1994-2024 Wordsmith

Powered by UBB.threads™ PHP Forum Software 7.7.5