v2.2.3 / chapter 6 of 13 / 01 jan 07 / greg goebel / public domain
* During World War II, Allied codebreakers enjoyed an even greater level of success than they had in World War I. The Americans deeply penetrated Japanese codes, while the British performed as or more impressive penetrations of German codes. When war broke out between the United States and Japan in late 1941, the Americans had been decrypting Japanese codes and ciphers since the 1920s, as an outgrowth of US codebreaking activities begun during the First World War. This chapter outlines the evolution of US codebreaking activities up to the beginning of World War II in the Pacific.
* The first formal US codebreaking organization was established by Herbert O. Yardley, who came to Washington DC from the state of Indiana in 1912, when he was 23 years old. He landed a job as a code clerk with the US State Department. He grew interested in encryption and read everything he could find on the subject. In 1916, Yardley received an encrypted message for US President Woodrow Wilson and decided to crack it as an amusement. To his shock, he cracked it in two hours. He continued to study the security, or rather the lack of it, of American codes and ciphers, and in 1917 presented a paper on the subject to his boss, David Salmon. Salmon was startled, but to ensure that Yardley wasn't simply blowing smoke, handed him a set of encrypted messages to crack. Yardley cracked them all in a few weeks.
The United States finally declared war on Germany on 6 April 1917. Yardley decided that the War Department would be a better path for advancement than the State Department, and enlisted in the Army in late June 1917. He was immediately put in charge of Military Intelligence Section 8 (MI-8), which was responsible for codes and ciphers. Yardley promptly built up an empire, establishing sections to improve the pathetic security of American codes and ciphers, and to crack the codes and ciphers of other countries.
When the Armistice was signed on 11 November 1918, ending the fighting in Europe, Yardley worried that he might be out of a job. However, the value of codebreaking organizations had been recognized by the US military and government, and less than a year later, in May 1919, Yardley was put in charge of an "American Black Chamber", as Yardley called it, jointly funded by the State Department and the War Department.
* The Black Chamber was set up in New York City, both to conceal its operations and because some odd law prevented the State Department from spending parts of its budget in Washington DC. Yardley set up a commercial business, the "Code Compilation Company", in the bottom floor of the brownstone building where the Black Chamber had its offices. The Code Compilation Company was partly a front and partly a way to obtain a little funding by selling cryptographic services to businessmen.
In 1919, Yardley managed to crack the Japanese diplomatic code. This information proved extremely valuable in assisting the Americans in negotiations during the 1921 Washington Conference on naval arms limitation. Unfortunately, after the conference, the need for the Black Chamber began to fade. The problem with justifying the organization's existence was complicated by the fact that the US government had passed the Radio Communications Act in 1912, which essentially outlawed "eavesdropping" on radio or cable traffic. The Radio Communications Act had been in abeyance during the war because of mandatory censorship measures, but now that the war was over, the act was back in force. Basically, the Black Chamber was an entirely illegal operation, although clandestine agreements were carefully negotiated with Western Union and other telegraph companies for access to diplomatic cable traffic.
In 1924, with the utility of the Black Chamber falling steadily, its budget was cut in half. Even more severe legal restrictions were imposed by the Radio Act of 1927, reducing the flow of intercepts once more. Finally, in 1929, the new Secretary of State, Henry L. Stimson, pulled all State Department funding for the Black Chamber. A story has persisted that Stimson justified the measure by saying: "Gentlemen do not read other gentlemen's mail." However, this is disputed by some sources, and Stimson's decision may have been actually driven by budgetary problems, though the illegal nature of the Black Chamber's activities may have been a factor as well.
The Black Chamber was formally shut down on 31 October 1929. This was a week after the Black Thursday stock-market crash that initiated the Great Depression in America. Yardley was out of a job, and the market for unemployed codebreakers was not good.
* In need of money, Yardley considered writing a book on the Cipher Bureau. He knew this was a breach of security, and in particular knew that if the Japanese found out that their diplomatic codes had been compromised, the would adopt new codes that would be much tougher to crack. Despite this concern, Yardley also felt he had an obligation to act as what a later generation would call a "whistle-blower", alerting the public to the feeble and dangerous state of American codes and codebreaking. He approached publishers on the prospect of selling such a book, and found a buyer.
Unfortunately, one of the other publishers Yardley had approached went straight to the government and told them what Yardley was doing. Yardley was given warnings. He ignored them. In the spring of 1931, three articles written by Yardley on the Cipher Bureau were serialized in THE SATURDAY EVENING POST magazine, leading up to the publication of Yardley's book, THE AMERICAN BLACK CHAMBER, on 1 June 1931. The book was a best-seller, and it was translated into several languages, including Japanese. Of course, the Japanese quickly decided to adopt improved encryption schemes based on cipher machines. The US government denied everything in Yardley's book. Government officials considered both arresting Yardley and suppressing his book, but decided the damage had been done and that taking action against him would only make things worse.
Yardley then decided to follow up his success with THE AMERICAN BLACK CHAMBER with a new, more scholarly book titled JAPANESE DIPLOMATIC SECRETS: 1921-1922. The government wasn't being taken by surprise this time. In 1933, the authorities told Yardley that if he persisted in trying to publish the new book, he would be arrested and charged with espionage. JAPANESE DIPLOMATIC SECRETS: 1921-1922 was classified and not released to the public until 1979. The whole fiasco led to the passage in 1933 of a US national-security law that still remains in force as "Section 952 of Title 18 of the United States Code", which threatens anyone who has been a US employee and publicly releases official diplomatic documents with up to ten years in prison. Yardley had been put in his place.
* Despite Yardley's concerns, the death of American codebreaking had been "greatly exaggerated". Months before Secretary Stimson closed the Cipher Bureau, in the spring of 1929 the US Army had decided to switch responsibility for cryptanalysis from the Military Intelligence branch to the Signal Corps, leading to the establishment of the "Signal Intelligence Service (SIS)" on 24 April 1930. The SIS sounded impressive on paper, with separate divisions for writing codes, cracking them, radio intercept and direction-finding, and development and production of invisible inks. In reality, the SIS was a very small and modestly-funded organization, but it was noteworthy for its head, William Frederick Friedman.
Friedman's parents were Hungarian Jews who had been living in Russia when he was born in 1891. Antisemitism drove the family out of Russia to the United States, and the family did well in the new country, sending their son to Cornell University. He graduated in 1914 with a degree in genetics, and got a job with Colonel George Fabyan, a wealthy and unconventional Illinois textile merchant who had a large estate outside of Chicago. His title of "Colonel" was a state militia rank, awarded to him by the governor of Illinois.
Fabyan hired Friedman to perform research into plant and animal breeding. Fabyan had other projects, and one involved a young woman named Elizebeth Smith. Smith was a librarian, and was hired to assist two sisters named Gallup, who were "Baconians", meaning they believed that Sir Francis Bacon had actually written the works of Shakespeare, leaving clues in the form of a subtle code in the plays. Friedman was an amateur photographer, and Fabyan enlisted him to take pictures of Shakespearian texts. Friedman became curious and began to study codes and ciphers on his own, becoming very proficient in the art. Elizebeth Smith also got interested in codes and ciphers and became an expert as well. Fabyan finally established a cryptanalytic team and put Friedman in charge of it.
The Colonel had a few government contacts that he wished to cultivate. He let them know that he had a codebreaking facility and would crack messages for the government for free. This was before Yardley had set up his MI-8 operation, and so at the time the US government had no formal codebreaking resources. The Colonel's contacts were definitely interested. In 1916, Fabyan began to receive encrypted messages from the government. Friedman's little codebreaking group cracked them and sent them back. When American involvement in the European war began to appear inevitable in early 1917, Fabyan decided to offer the services of his little cipher group to the US War Department. The War Department send Lieutenant Joseph Mauborgne, one of the few people there who knew anything about the subject, to investigate, and Mauborgne was thoroughly impressed.
This was a busy time for William Friedman. He married Elizebeth Smith in May 1917, and in the fall of that year, taught a six-week cram course in cryptanalysis to a handful of military officers. The course went over so well that he taught the same course to a much larger group of officers in early 1918, with a follow-up session later in the spring. By June 1918, Friedman was in the Army, which sent him to France to break German codes. He returned home and was mustered out in April 1919. He tried to find work as a geneticist for a while, but failing in that, ended up working for Colonel Fabyan again.
* Friedman was writing down his ideas about cryptanalysis, publishing seven pamphlets, collectively known as the "Riverbank Publications" after Colonel Fabyan's Riverbank estate. All were original and insightful, but the seventh pamphlet, titled "The Index Of Coincidence And Its Applications In Cryptography", was the most significant. In this pamphlet, Friedman considered the problem of trying to decrypt a Vigenere cipher with a long random key. His analyses showed that in any long string of English text, any two consecutive characters selected from the plaintext would be the same character 7% of the time on the average, due to the common nature of digraphs such as "ee" or "oo" or "tt". In contrast, in purely random text, the same character would only occur twice 4% of the time on the average.
Suppose Holmes has two ciphertexts encrypted by a Vigenere cipher. If the cipher system is implemented properly, the percentage of digraphs consisting of the same two characters in either ciphertext is 4%.
Now further suppose Holmes compares two ciphertexts that he believes may have common key sequences at some location or another, but not necessarily the same location in each. He arranges the two ciphertexts side by side and then obtains statistics on how often the letters in both ciphertexts are the same. If the two ciphertexts don't have a matching key, the coincidences will be random, and matches will occur only 4% of the time.
Holmes then shifts the relative positions of the two ciphertexts and performs the analysis again. If the result of the analysis shows that the matches are again about 4%, then he knows the two ciphertexts still don't have the same key.
Holmes keeps on doing this, until he finally shifts the two ciphertexts into a relative position where they have a key that matches. The percentage of duplications will then jump to 7%. Once this happens, Holmes can use one ciphertext as a crosscheck against cryptanalysis of the other ciphertext by conventional means, in much the same way that a one-time pad cipher can be broken if two messages are encrypted with the same cipher key. This is certainly another "word puzzle from Hell" exercise, but it does give him a lever to crack open the cipher.
This analysis and related ones Friedman performed in the same pamphlet took the use of statistical tools for cryptanalysis to an entirely new level, and would eventually establish Friedman as one of the most significant figures in cryptology.
* Working for Fabyan again had proven troublesome. Fabyan had lured Friedman back with big promises that hadn't been kept. Fabyan was also self-important, for example trying to take credit for Friedman's pamphlets. Friedman was eager to find work elsewhere. In November 1920, the Friedmans accepted positions as civilian cryptographers for the US government and moved to Washington DC. William took a job with the Army Signal Corps, where he became the chief cryptologist, while Elizebeth worked briefly for the Navy, and then settled down as a housewife to raise a family. She would perform part-time work for the Treasury Department, and during Prohibition cracked ciphers used by rum-runners.
While the Friedman family lived a comfortable middle-class life, William continued to advance the state of the art in devising and breaking ciphers. In 1923, as a consequence of teaching courses on cryptology to US Army personnel, he published a short but classic text on the subject titled ELEMENTS OF CRYPTANALYSIS. The little book was very clear and concise and proved influential. In particular, Friedman used it to lay down precise definitions of cryptologic terms. Up to this time, the terminology used in the field was inconsistent and confusing, but Friedman's definitions cut through the confusion so well that they became the accepted terms. Even the terms of "cryptanalysis" and "cryptology" as used in this document owe their origins to Friedman's book.
* Through the 1920s, Friedman lived a modest existence working for the Army. At the time, he was the only cryptanalytic resource the War Department owned, and only help was a retired prize-fighter who was his clerk and typist.
With the establishment of the SIS, Friedman acquired greater responsibilities and authority, but not by leaps and bounds. He was authorized to hire am additional staff of four junior cryptanalysts and an assistant cryptanalytic clerk. The pay for a junior cryptanalyst position was only $2,000 USD a year, which was modest even for the time, but the Depression was on and jobs were scarce. Friedman managed to hire three highly skilled mathematicians who also had useful language skills: Frank B. Rowlett, who spoke German; Abraham Sinkov, who spoke French; and Solomon Kullbach, who spoke Spanish.
Friedman expected to have a difficult time finding someone with fluent Japanese language skills to fill the fourth slot. An acquaintance in the government called to request a job interview for his nephew, John B. Hurt. Friedman accepted out of courtesy and arranged the interview, with little expectation that Hurt would be worth the bother, but the Army officer who conducted the interview was astonished at Hurt's fluency in Japanese.
With the hiring of an assistant clerk, the SIS was fully staffed with seven people. Until 1937, the yearly budget of the organization never exceeded $17,400 USD. Along with cracking coded messages, working from intercepts obtained by a few scattered Signal Corps radio listening posts, the SIS also trained Army officers in cryptology.
In August 1935, Major Haskell Allison replaced Friedman as administrative head of the SIS, freeing the cryptologist to focus on technical issues. Friedman gave his people the full benefit of his two decades' experience as a master of codes and ciphers. As international tensions continued to rise through the 1930s, the SIS grew accordingly, but Friedman and his four disciples remained the prime movers.
* By the mid-1930s, Japanese aggression in China was creating antagonism with the United States, and a conflict between Japan and America seemed increasingly possible. In 1937, under the direction of Joseph Mauborgne, now a major general and as of October of that year the Army's Chief Signal Officer, the SIS began to grow, acquiring more personnel and linking into an expanding network of Signal Corps listening posts in the US and overseas. When Hitler invaded Poland on 1 September 1939, the SIS had a staff of 19; by the time of the Japanese attack on Pearl Harbor on 7 December 1941, it had a staff of 331; and within a few years, it would have a staff of thousands.
However, in 1937, the SIS was still constrained by laws that made its activities illegal, but on 30 March 1938 US War Secretary Harry H. Woodring signed an order authorizing the work of the SIS. The laws remained in force, but if anyone wanted to make an issue of it, Friedman and his people would no longer be held liable.
In 1932, the Japanese had introduced a new "Type A" cipher machine to encrypt top-level diplomatic communications. The SIS made cracking Type A a top priority in 1935 and cracked it in 1936. After cracking the Type A machine, Friedman began to worry about US Army correspondence mentioning the Type A machine falling into Japanese hands, and so assigned the machine the codename "RED". Some sources mention an "ORANGE" cryptosystem that preceded RED, but details are painfully vague and contradictory, and ORANGE may simply be the name given to the Japanese diplomatic ciphers cracked by Yardley's Black Chamber.
RED was a relatively primitive machine, and SIS cryptanalysts managed to crack it using the statistical analysis techniques pioneered by William Friedman. Frank Rowlett devised a simple code wheel device to allow RED messages to be quickly deciphered. However, the Japanese gradually recognized that RED was vulnerable, and so in the spring of 1939 they introduced the much more formidable "Type B". The Americans named the machine "PURPLE" and set out to crack it.
* The nature of the Japanese written language adds some complications to the design of a code machine that wouldn't apply to Western languages.
At the core of the Japanese written language is the "kanji", which is basically the Chinese ideographic symbol set. About 2,000 kanji are needed for minimal Japanese literacy, with the full set amounting to more than 20,000 symbols, though few Japanese know them all.
The kanji are supported by much smaller phonetic set known as "kana". The kana describe the sounds used in Japanese pronunciation, and they can be written in two styles known as "hiragana" and "katakana". A limited number of Japanese words are written in hiragana, but hiragana characters are mostly used as modifiers to adapt the Chinese characters to Japanese syntax, the two languages actually having little in common besides the character set. Katakana characters are used to write down foreign words for which no kanji exists. Both are used in a Japanese dictionary, in different contexts, to give the pronunciation of kanji symbols. The two sets of kana characters are equivalent; they just look different. There are 50 basic kana characters, with about 20 additional modifications.
The Japanese also adopted the Western Roman character set, logically calling it "Romanji", using it for stylish titles and particularly for acronyms, which don't really make sense in the traditional Japanese symbol set. Since kanji was too cumbersome for a cipher machine, that left only the alternatives of sending kana symbols or spelling out the same set of sounds using Romanji letters.
The Japanese diplomatic machines sent messages in Romanji. Japanese is very context-sensitive, with many words that sound the same, but transmitting the language in Romanji using phonetic spellings is conceptually no different than speaking it over the telephone: as long as some care is taken to avoid ambiguities, there should be little confusion. Although writing Japanese phonetically in Romanji was the least obvious approach -- it's not something a Japanese person would normally do -- using Romanji had the advantage that Western documents could be sent verbatim, a useful capability for an embassy.
PURPLE's encryption scheme was along the lines of that of the Enigma machine, with three banks of scramblers, an input keyboard, and a set of output indicators, but its construction was very different. Instead of rotors, it used telephone "stepping switches", a type of electromechanical switch used to establish telephone connections in a telephone exchange. The basic element was a switch with a single input lead and a moveable "wiper" arm connecting it to 25 possible output leads. The wiper arm was moved by an electromagnetic solenoid; the solenoid would move the arm from position to position until it reached the last position, and then a spring would pop the arm back to the first position.
Building a cipher machine with such switch elements is conceptually straightforward. The 25-contact arrangement immediately suggests encrypting 25 letters, which as shown in previous chapters is adequate for encrypting a Roman alphabet. Three banks of 25 switch elements each, for a total of 75 switch elements, can be wired together, with each bank performing one level of scrambling.
The 25 switch elements in each bank all begin in the same position and advance together in unison when activated by their solenoids. Each has 25 output leads. Of course, since the next bank can only accept 25 input leads, the output leads of the 25 switch elements in a bank are wired together. This issue of wiring is complicated to explain.
Suppose we consider a simplified or "toy" scheme with a bank of only four
switch elements labeled "A" through "D", with each switch having only four
positions, "1" through "4", and connected to a second bank of four switch
elements labeled "W" through "Z". We can then represent wiring, say, leads
A1, B1, C1, and D1 together and connecting them to the input of W as follows:
W = A1 + B1 + C1 + D1
The simplest possible wiring arrangement is as follows:
W = A1 + B1 + C1 + D1
X = A2 + B2 + C2 + D2
Y = A3 + B3 + C3 + D3
Z = A4 + B4 + C4 + D4
This is also the most absolutely useless arrangement: since the wipers of
all four switch elements move in step to the same positions, they would all
short each other out, all the time. The rule is that a specific lead on one
switch element cannot be connected to the same lead on another switch element
-- for example A1 can't be wired to C1, B3 can't be wired to D3, and so on.
A more workable wiring pattern would be of the form:
W = A1 + B4 + C2 + D3
X = A2 + B3 + C4 + D1
Y = A3 + B2 + C1 + D4
Z = A4 + B1 + C3 + D2
-- or, resorted by wiper position instead of switch element:
W = A1 + C2 + D3 + B4
X = D1 + A2 + B3 + C4
Y = C1 + B2 + A3 + D4
Z = B1 + D2 + C3 + A4
This resorting reveals the connection routing for each wiper position. In
the first position, WXYZ = ADCB; in the second position, WXYZ = CABD; in the
third, WXYZ = DBAC; and in the fourth, WXYZ = BCDA. Another workable wiring
arrangement would be as follows:
W = A4 + B3 + C1 + D2 = C1 + D2 + B3 + A4
X = A2 + B1 + C3 + D4 = B1 + A2 + C3 + D4
Y = A1 + B4 + C2 + D3 = A1 + C2 + D3 + B4
Z = A3 + B2 + C4 + D1 = D1 + B2 + A3 + C4
Given 25 switch elements with 25 elements each, there will be 25 different
sets of wirings, each giving a mixed substitution cipher alphabet for one
position of the wipers. This gives an advantage over a rotor. A rotor
implements a single substitution cipher alphabet that is shifted for each
letter enciphered, and for each rotor a codebreaker only has to figure out
the layout of one set of wires. In the case of this stepping switch cipher
machine, for each switch bank the codebreaker has to figure out 25 sets
of wires.
Since it would be nice to be able to use the same machine for encryption and decryption, the output of the final bank is wired back into itself to provide reciprocal operation, just as with the Enigma. As with the Enigma rotors, the switches in each bank step at a different rate: one bank switches with each letter, another bank switches every 25 letters, and the third bank switches every 25 * 25 = 625 letters. This gives a sequence length of 25 * 25 * 25 = 15,625.
This gives a fairly workable cipher machine. Of course, the switch banks
can't be yanked out like rotors and rearranged, but it is possible to produce
an equivalent effect by changing which banks step at which rate using a
master selector switch. With three banks, there's six possibilities; the
table below gives the arrangements of banks "B1" through "B3" for selector
switch positions "SW1" through "SW6", with the banks selected to step by 1,
25, or 625:
B1 B2 B3
______________________
SW1 1 25 625
SW2 1 625 25
SW3 25 1 625
SW4 25 625 1
SW5 625 1 25
SW6 625 25 1
______________________
Another improvement would be to add a plugboard to rearrange the inputs
between the keyboard and the scrambler system, much like the Stecker on the
Enigma machine. This idealized cipher machine's setup would involve wiring
up the plugboard, setting the stepping sequence for the banks, and also
setting the initial positions for the wipers in the banks. The initial
positions could be set up by using the selector switch to specify a step of 1
for each of the three banks in turn, with the switch wipers advanced by
simply enciphering an arbitrary letter and not bothering to send it.
* This idealized cipher machine is along the lines of a PURPLE machine, but PURPLE was designed with real-world constraints such as size and available technology in mind. One of the constraints was that the stepping switches were only available in modules of six switches each.
It is somewhat difficult to figure out exactly what was going through the mind of the engineer that designed the PURPLE machine, since it seemed as though with a little bit of effort he might have designed a more obvious machine, along the lines of that described above, that was more secure. No doubt he had to deal with such non-cryptographic issues such as the size and cost of the machine, though it seems plausible that the structure of romanized Japanese phonetics was a factor in some way as well.
PURPLE enciphered a full alphabet of 26 letters, but the plugboard only allowed shuffling around 20 of them. These 20 letters were run through three banks of switch elements, using four switch modules in each bank to provide 24 switches; what was done with the "extra" four switches is unclear. However, the remaining 6 letters were run through a single bank, composed of one switch module. It is also unclear if PURPLE was designed for reciprocal operation, though it is difficult to figure out what else would have been done if it wasn't. Incidentally, such details as are available about the earlier RED machine seem to imply it used a rotor-type scheme instead of telephone switching banks, with a similar division of letters and a much shorter stepping sequence.
* Although PURPLE was much more difficult to crack than RED, the SIS group had some advantages. As mentioned, PURPLE had some similarities to RED, and during the period when Japanese embassies were upgrading to the new code machine, the same messages were foolishly sent in both RED and PURPLE. Decryption was further helped by the fact that the format of the contents of the messages was very uniform. Certainly this is how the Japanese do things, but it is possible that the uniform format was partially imposed by the necessity to reduce the ambiguity of messages sent phonetically rather than in kanji.
The SIS use punched card "tabulating machines" that could perform simple sorts and other operations on data typed into punch cards, assisting in statistical analyses of Japanese traffic. While working on the RED code, Friedman had obtained an IBM punch card machine that the US Army Quartermaster Corps had used for accounting purposes and was phasing out. The punch card machine had proven so useful that Friedman bought another one.
In the spring and summer of 1940, with war in full gear in Europe, PURPLE traffic increased dramatically, particularly for communications between Tokyo and the Japanese embassies in Berlin and Rome. These communications led to the "Tripartite Pact", which established the "Berlin-Rome-Tokyo Axis". The large flow of encrypted data gave SIS an opportunity to obtain a better grip on PURPLE. On 27 September 1940, the Tripartite Pact was signed. Two days previously, the SIS had been able to adequately read a non-trivial message encrypted in PURPLE for the first time.
Being able to crack PURPLE messages was only half the job. US intelligence needed to know what the messages contained as soon as possible, and that meant duplicating the Japanese cipher machine. Remarkably, the SIS managed to reverse-engineer the PURPLE output and build an equivalent cipher machine. With the "PURPLE analogue" machine available, the SIS was able to decode Japanese diplomatic messages as quickly as Japanese code clerks themselves.
Since the SIS didn't have the funds to build the machines themselves, Friedman lobbied the US Navy for help, and five of the PURPLE analogue machines were produced. Although no actual Japanese PURPLE code machines survived the war, since the Japanese were strict about procedures and procedure was to completely destroy the code machines when pulling out of an embassy, fragments of a code machine found in the Japanese embassy in Berlin after the war suggest that the PURPLE analogue machine was remarkably accurate.
The difficulty of managing a greatly expanded organization and the prospect of war put Friedman under intense pressure. In January 1941, he was admitted to the Neuropsychiatric Ward of the Walter Reed Army Hospital in Methesda, Maryland. He was back on the job by 1 April, but never really carried a full workload again. His insights were unsurprisingly still regarded as extremely useful.
* As a footnote to RED and PURPLE story, the Imperial Japanese Navy (IJN) also used cipher machines based on commercial telephone-switch banks. The first was the "CORAL" machine, which was used by naval diplomatic attaches. It enciphered Japanese written phonetically in Romanji, fed through three levels of switching elements, without any division of the letter set as was done in PURPLE and RED. CORAL apparently had a plugboard, but the selector switch could only allow the operator to choose one of three bank-stepping arrangements, not six as with PURPLE.
The "JADE" machine was actually used tactically by the IJN from late 1942 into 1944. JADE was very similar to CORAL, the main difference being that JADE actually enciphered kana directly, using a shift key to allow entry of the complete set of 50. Since it was not used by diplomatic officials, there was no reason to worry much about sending Western documents with it. CORAL and JADE were both apparently cracked, but the details of these exercises are hard to find. It appears that these ciphers were never of more than secondary importance. JADE machines were captured intact during the war.
* The US Navy developed a crytanalysis organization in parallel with the Army SIS. During the First World War, cryptanalysis was handled by the Code & Signal Section of the Naval Communications Service. In July 1922, the Navy cryptanalysis group was formalized as "Office of Chief Of Naval Operations, 20th Division of the Office of Naval Communications, G Section / Communications Security", or much more simply, "OP-20-G". In January 1924, a 34-year-old US Navy lieutenant named Laurence F. Safford was assigned to expand OP-20-G's domain to radio interception. He worked out of Room 2646, on the top floor of the Navy Department building in Washington DC.
Japan was of course a prime target for radio interception and cryptanalysis, but there was the problem of finding personnel who could speak Japanese. The Navy had a number of officers who had served in a diplomatic capacity in Japan and could speak Japanese fluently, but there was a shortage of radiotelegraph operators who could read Japanese Morse code communications sent in kana. Fortunately, a number of US Navy and Marine radiotelegraph operators operating in the Pacific had formed an informal group in 1923 to compare notes on Japanese kana transmissions. Four of these men became instructors in the art of reading kana transmissions when the Navy began conducting classes in the subject in 1928.
The classes were conducted by the Room 2646 crew, and the radiotelegraph operators became known as the "On-The-Roof Gang". By June of 1940, OP-20-G included 147 officers, enlisted men, and civilians, linked into a network of radio listening posts as far-flung as the Army's.
OP-20-G did some work on Japanese diplomatic codes, but the organization's primary focus was on Japanese military codes. The US Navy first got a handle on Japanese naval codes in 1922, when Navy agents broke into the Japanese consulate in New York, cracked the safe, took photographs of pages of a Japanese navy codebook, and left, having put everything back as they had found it.
From this start, the US Navy was consistently able to keep up with changes in Japanese codes. In September 1930, the Japanese Navy conducted "Grand Maneuvers" that involved most of their increasingly powerful fleet. US Navy decrypts of messages sent during this military exercise gave the Americans a remarkably detailed picture of Japanese naval strength.
* Although Laurence Safford had spent two stints in sea duty after working in OP-20-G, he had finally returned to the organization in 1936 to become its head. By the fall of 1941, OP-20-G was growing in much the same way as the SIS.
The Navy cipher bureau operated out of three main bases: headquarters in Washington DC, a section at Pearl Harbor in Hawaii, and a section in the fortified caves of the island of Corregidor, in the Philippines. The codebreakers were backed up by a far-flung network of listening and radio direction finding stations.
The US and Japan were clearly headed for a crisis, and Navy communication intercepts from Hawaii of Japanese naval transmissions indicated that the Japanese fleet was assembling for a major operation. The details were unclear.
The SIS and OP-20-G were badly hobbled by bureaucracy. Part of the problem was that they had unsurprisingly become rivals, competing with each other to provide their intelligence date, codenamed "MAGIC", to high officials. Both organizations ended up seeming foolish and obnoxious, and the word came down that the two groups were to cooperate. That was easier said than done, and rivalries between the two cryptanalysis teams would remain a problem for a long time. The best that SIS and OP-20-G were able to do was come to an agreement in 1940 to provide MAGIC intelligence on alternating days, and try to draw up some vague guidelines for which team handled what traffic. Complicating matters was that the US Coast Guard, the US Federal Bureau of Investigation, and even the US Federal Communications Commission also had radio-intercept operations.
The result was that much of the MAGIC intelligence was wasted. There was no efficient process for assessing and organizing the intelligence, or getting it to its proper end users. This was a dangerous problem as the time was rapidly approaching when that intelligence was a matter of life and death.
* In the dark hours of the morning of 7 December 1941, the US Navy communications intercept station on Bainbridge Island, in the state of Washington, picked up a radio message being sent by the Japanese government to the Japanese embassy in Washington DC. It was the last in a series of 14 messages that had been sent over the previous 18 hours.
The messages were decrypted by a PURPLE analogue machine at OP-20-G and passed to the SIS for translation from Japanese, early on the morning of December 7. Army Colonel Rufus Bratton and Navy Lieutenant Commander Alvin Kramer independently inspected the decrypts.
They both became alarmed. The decrypts instructed the Japanese ambassador to Washington to inform the US Secretary of State, Cordell Hull, at 1:00 PM Washington time that negotiations between the US and Japan were ended. The embassy was then to destroy their cipher machines. This sounded like war, and although the message said nothing about any specific military action, Kramer also realized that the sun would be rising over the expanses of the central and western Pacific by that time. The two men both tried to get in touch with Army Chief of Staff General George C. Marshall.
After some agonizing delays, Marshall got the decrypts and methodically examined them. He realized their importance and sent a warning to field commanders, including Major General Walter Short, the Army commander in Hawaii. However, Marshall was reluctant to use the telephone because he knew that telephone scramblers weren't very secure and sent it by less direct channels. Due to various constraints and bumblings, Short got the message many hours after the Japanese bombs had smashed the US Navy's fleet at anchor in Pearl Harbor.
In the aftermath of the attack, Henry L. Stimson, now the US Secretary of War, ordered a review of the ramshackle communications intelligence structure that had contributed to disaster at Pearl Harbor. By the spring of 1942, resources were being poured into the SIS and other intelligence organizations, and bureaucratic obstacles were being knocked down. There was a war on.
