Oral-History:Claude E. Shannon

About Claude E. Shannon

Claude E. Shannon was born in Gaylord, Michigan, on April 30, 1916. He received the B.S. degree in Electrical Engineering and Mathematics from the University of Michigan in 1936. From 1936 to 1940, he attended M.I.T., combining graduate studies with work experience. For two years he was a research assistant in the Electrical Engineering Department, where he operated Vannevar Bush's electromechanical differential analyzer. He was an assistant in the Mathematics Department from 1938 to 1940, and during 1939-1940 was a Bolles Fellow. He received the S.M. degree in Electrical Engineering and the Ph.D. degree in Mathematics from M.I.T. in 1940.

Shannon was associated with the Institute for Advanced Study at Princeton University for one year through a 1940-1941 National Research Fellowship. Beginning in 1941, he served as a research mathematician for Bell Telephone Laboratories in Murray Hill, N. J. Shannon also served as consultant to the National Defense Research Committee.

Dr. Shannon's work included the following fields: the use of Boolean algebra in relay and switching circuits, communication theory, mathematics of cryptography, theory of differential analyzer, and the use of computing machines for numerical operations. He also studied chess-playing and maze-solving machines (which he also built), the theory of Turing machines, design of reliable machines from unreliable components, stochastic processes, the algebra of genetics, and graph theory.

In 1940, Dr. Shannon was the recipient of the Alfred Noble Prize of the American Institute of Electrical Engineers for his work in switching theory. He received the Morris Liebmann award of the Institute of Radio Engineers in 1949 for his communication theory work. Yale University awarded him an honorary Master of Science degree in 1954, and in 1955, Dr. Shannon received the Stuart Ballantine medal of the Franklin Institute for work in communication theory. He was the author of approximately thirty-five technical papers, and held several patents. He was co-author, with Warren Weaver, of The Mathematical Theory of Communication, and co-editor, with John McCarthy, of Automata Studies.  Dr. Shannon was a Fellow of the Institute of Radio Engineers. He died in 2001.

This interview concentrates on Claude Shannon's career in the 1940s. Interviewer Robert Price focused on Shannon's working relationship with Norbert Wiener and on the intellectual genealogy of information modeling. Shannon's comments narrate his career progression from fellowship work for the National Research Council and Institute for Advanced Study to World War II-era employment with Bell Labs. Shannon identifies influences on his information theory work and considers his own influence on Norbert Wiener. He analyzes his 1945 cryptography report and its 1949 publication as Communication Theory of Secrecy Systems. Assessing the security concerns that limited professional communication about cryptography during World War II, Shannon explains his access to the work of John Tukey, William R. Bennett, John Riordan, and Alan Turing. The interview details Shannon's interactions with Turing at Bell Labs and in England, before and after the war. In response to Price's questions about intellectual influences, Shannon concludes that problem-solving, rather than concern with the work of his contemporaries, drove his research process. The concluding section of this transcript considers the relevance of Shannon's work to military jamming and anti-jamming applications and to spread spectrum technologies. Based on his own research experiences at M.I.T.'s Lincoln Laboratory, interviewer Robert Price provides insight on the influence of Shannon's noise carrier work.

About the Interview

CLAUDE E. SHANNON: An Interview Conducted by Robert Price, 28 July 1982

Interview # 423 for the IEEE History Center, The Institute of Electrical and Electronics Engineers, Inc.

Copyright Statement

This manuscript is being made available for research purposes only. All literary rights in the manuscript, including the right to publish, are reserved to the IEEE History Center. No part of the manuscript may be quoted for publication without the written permission of the Director of IEEE History Center.

Request for permission to quote for publication should be addressed to the IEEE History Center Oral History Program, IEEE History Center at Stevens Institute of Technology, Castle Point on Hudson, Hoboken, NJ 07030 USA or ieee-history@ieee.org. It should include identification of the specific passages to be quoted, anticipated use of the passages, and identification of the user.

Original Copyright: 1982 Robert Price Documentary Trust. 2003: Copyright transferred to the IEEE.

It is recommended that this oral history be cited as follows:

Claude E. Shannon, an oral history conducted in 1982 by Robert Price. IEEE History Center, Hoboken, NJ, USA.

Interview

Interview: Claude E. Shannon

Interviewer: Robert Price

Date: 28 July 1982

Location: Winchester, MA

Discrete and continuous mathematics; Norbert Wiener

Price:

[Beginning of transcript]...and Claude’s autograph [on R. Price’s copy of the NDRC Division 7 Summary Technical Report, Volume 1, Gunfire Control] on his prediction and smoothing theory work for the war. That reminds me that this is continuous time stuff. I remember Norbert Wiener saying that you loved the discrete and eschewed the continuum, but it sounds like you were working in both discrete and continuous mathematics at the same time.

Shannon:

Yes, well he, Norbert, got that notion because of my thesis which is highly discrete, the Boolean algebra and all that.

Price:

And also the genetics.

Shannon:

That too, yes. I do not eschew the continuum. [laughs ]

Price:

Well, this proves it. Maybe he wasn’t too familiar with your exposition of his theory during the war. You studied his theory but you didn’t communicate with him directly, is that right, during the war?

Shannon:

Well, it only appeared publicly after the war. You just told me in 1948, thus . . .

Price:

Yes, but Classified published in 1942, and you started studying it as soon as it appears as a Classified document.

Shannon:

Oh, I read his "Yellow Peril," a nickname, published by Wiley in 1949 as The Interpolation, Extrapolation, and Smoothing of Stationary Time Series and referred to . . .

Price:

That’s what I mean.

Bell Labs anti-aircraft group

Shannon:

And we worked on, at Bell Laboratories, a group under [Hendrik] Bode was involved in anti-aircraft control. Anti-aircraft directors.

Price:

This is gunfire control.

Shannon:

Right. And it included people like Stibitz and Blackman and Bode, and myself and others, McNair . . . . Yes, most of the mathematical group was in that. It took two directions. There were two concepts [as] to how to aim a gun. One is . . . you have some telescopes tracking this, this airplane. Let’s say you can get three coordinates out of this, and these are noisy coordinates because the plane is moving around and the readings are not good anyway. You want to aim far in the future, I mean fractions of a mile ahead, because it takes a long time for the shell to get there at the time of the plane getting there, and they’ve got to figure just where. There were a couple of philosophies. One was just trying to assume that the plane was going on the straight line course and try to smooth the data that you had. So you could use a thing like Wiener’s smoothing or Kolmogorov[’s], a Russian who was also involved.

Price:

Say smoothing a polynomial in this case?

Shannon:

Or taking the three coordinates and running them through a smoothing network, or a smoothing and derivative taking network to try to extrapolate into the future.

Price:

Well, if you’ll pardon me, I think I’m more interested in not so much the technology, but your relations with Norbert Wiener in the context of this technology. You saw the "Yellow Peril," you wrote your own version of it here (showingGunfire Control), an exposition. Did you ever have any contact with Wiener during the war to give him any feedback on your studies?

Shannon:

Not that I recall. Not at a scientific level. I’m sure I met him and talked to him from time to time.

Modeling information as a random process

Price:

For a long time I was under the misapprehension that you had been a student of Wiener's before the war, but that was never the case. You were in the same department together, right, and you must have seen each other, but you were never a student?

Shannon:

I was a student in one class. I took a course in Fourier analysis.

Price:

I see, and where were you?

Shannon:

I didn’t have him as a doctoral student.

Price:

No, I see. But then where did you get the idea that information could be modeled--I mean, when did you get it? You got it from Wiener, I believe . . .

Shannon:

No, I didn’t.

Price:

. . . that information could be modeled as a random process?

Shannon:

No.

Price:

Where did that come from then? Did it come out of cryptography?

Shannon:

No.

Price:

Because you know, he generally has the attribution for that, he has the credit for that modeling of information as a random process.

Shannon:

Well, I don’t believe his model.

Price:

Noise is a random process, but the idea that information would be a random process . . .

Shannon:

Well, I don’t know. I hadn’t even heard about the statement that you just made.

Price:

That he generally gets the credit for modeling information as a random process?

Shannon:

No.

Price:

Well, you actually have that as a footnote [on pp. 52-53] in your 1948 publication, A Mathematical Theory of Communication. It may have been a . . .

Shannon:

What does it say?

Price:

OK, I have that here because I want to get that autographed too. I thought that . . . that did come from Wiener, but that he had never carried it very far. It was just a philosophical notion of his, I thought.

Shannon:

Well, this doesn’t say anything about information at all.

Price:

No, no, it certainly doesn’t.

Shannon:

This is the Ergodic Theorem, which has to do with averaging over space as contrasted with averaging in time.

Price:

Okay, I didn’t remember that too accurately. Of course, in the early 1930s he did the Generalized Harmonic Analysis for stochastic functions. But he didn’t at that time say, I guess, that a stochastic function could be a model of an information process.

Shannon:

I don’t think he ever did.

Price:

Did he never say it before the book Cybernetics: Or Control and Communication in the Animal and the Machine?

Shannon:

Not that I know of.

Price:

I see. Well, would you be suggesting that maybe it was the other way around?

Shannon:

Oh, I think so. I do indeed.

Price:

Well, it’s very important to have this on tape then. So now, it wasn’t given to you even by cryptography? I mean, when did it occur to you that information could be modeled as a random process?

National Research Fellowship at Princeton; switching publications

Shannon:

Well, during the entire period between 19 . . . well, let me start this way. I took this National Research Fellowship at Princeton in 1940-41, after I had got my doctorate at M.I.T., I applied for a National Research Fellowship.

Price:

In genetics, right? I have the title of your proposal and it was to work with Weyl in genetics.

Shannon:

With Weyl . . .

Price:

And Vannevar Bush had encouraged you to do that.

Shannon:

I worked with Weyl, yes, but I worked on Information Theory.

Price:

You may have done that, but ostensibly you were supposed to work on genetics, because that’s what Vannevar Bush had touted.

Shannon:

[laughs]

Price:

The title of your proposal as it is at the National Research Council is in genetics.

Shannon:

Audio File

MP3 Audio
(423 - shannon - clip 1.mp3)

Yes, well at any rate, when I got down to Princeton, well, let me try to recall . . . I applied for this, as I was getting my doctorate degree. And that very summer I spent at Bell Laboratories, I took a summer job there for three months, Thornton Fry came up to M.I.T. and to see various people and talked to me, and offered me a summer job because of my work on switching. And so I went down and I lived in Greenwich Village for that summer and I wrote a paper on two-terminal switching circuits which was not published until 1949, nine years later. And I worked for B. D. Holbrooke, who was a very bright switching man. And he set me to designing a circuit for a crossbar system, using a special tricky relay. But anyway, I worked on that about the same time I worked on this other thing, sort of unknown to Holbrooke, and wrote that up as a second paper. So that summer I did these two papers, and the second one became quite well known in switching circles. And I hate to rattle on, so many years after this I kept getting papers from Russia because they got all excited about this, I proposed this problem, what is the minimum number . . . the maximum/minimum . . . the minimum number of contacts required for the worst thing involving N relays, the worst of switching functions. I put some bounds on it and I came out with improved bounds and . . . .

Price:

That’s interesting because I think Brockway McMillan told me you were always interested in minimax problems.

Shannon:

Yes, I loved that —

Price:

I’d like to come back to that, but still the modeling of information as a stochastic process was beyond the ken of a communications engineer. You know, in those days you’d never have the vision to imagine that —

Shannon:

Well, let me get back to my story. It says, “I digress, I digress.” [laughter ] Anyway [laughter ], . . . when I went down after the summer digression at Bell Laboratories, I went to take up this National Research Fellowship in Princeton, and to work under Weyl. And when I went to him I said here’s what I really want to work on, and I talked about information and the measurement of information and how much it requires. And I told him I had already read Hartley’s paper, and that was an influence in my life. I had read that already, I think back in the University of Michigan, but I’m not sure, maybe it was as much as, anyway, several years previous. But it had struck me as an important paper, and I remember talking to Hermann Weyl about the relation of this to the Uncertainty Principle. Weyl was one of the great mathematicians of our time, actually. And he in fact had written papers on the Quantum Mechanics and the relation of the width of a spectrum against the time of a function. So the product relation . . .

Price:

Yes, Heisenberg Uncertainty.

Shannon:

Yes, the mathematical analog of that is a . . . sort of a radius of gyration of the one, times the [radius of gyration of the] other, has to be a certain size.

Price:

You said the right thing to him, to get him to approve your digression.

Shannon:

He thought that was a great thing too. [laughter] And he couldn’t care less about genetics, I think. He just took me on because people recommended me.

Price:

Of course, that thing you’re talking about is what Dennis Gabor pursued for such a long time and led to a dead end, so it’s sort of ironic that you would propose that, and Gabor you know, just massaged that Indeterminacy Principle, but you went so far beyond that.

Shannon:

Well, Gabor, we contact Gabor as we continue the story of my life . .. [laughter]

Price:

Sure. I mention it, it seems sort of interesting.

Shannon:

Yeah, up there you’ll see a hologram which has much to do with Gabor, but you don’t see it now, it’s not turned on. Anyway, Gabor had a lot to do with holograms.

Price:

Yes, well, that’s a Nobel Prize after all.

World War II gunfire control and mathematical analysis

Price:

So then, actually, Peter Elias had told me that you were working on Information Theory as early as 1940, and had been working at the Institute for Advanced Study, although I couldn’t find any trace of it in your proposal, and you didn’t leave any document behind —

Shannon:

What proposal?

Price:

For your proposal for your Fellowship.

Shannon:

Oh, the Fellowship.

Price:

And you didn’t leave anything behind you when you left the Institute apparently, you didn’t leave any files.

Shannon:

Oh, there’s another story. . .Betty [Mrs. Shannon] is learning more about my life--

Price:

Or does something exist?

Shannon:

[more] than she ever knew. Well, now what happens here. So I worked away at this, and oh, pretty soon, who appears on the scene but Thornton Fry, from Bell Laboratories, and the war is heating up. This is, you were just a child at that time, no doubt. But anyway.

Price:

Eleven years old.

Shannon:

Eleven, yes. Anyway, the war is heating up, and we’re not in the war yet, but the war in Europe and the American involvement is heating up. And, you know, Roosevelt is saying, we’ve got to have thousands of airplanes and guns and this and that, and all the scientists are getting badgered into building things.

Mrs. Shannon:

They’re drafting people like crazy.

Shannon:

And Thornton Fry was put at the head of a committee to study anti-aircraft control and anti-aircraft devices and things like that.

Price:

Yeah, concerned with the book you just autographed for me.

Shannon:

Right. He’s somewhere on the —

Price:

Yes, Division Seven, Gunfire Control.

Shannon:

Right. Gunfire control. And so he comes down and says, could I do some spare time work, he was impressed with my work that previous summer. [laughs ] So he says could you join my committee and work on this important subject for the national defense effort and this and that?

Price:

While carrying on your Fellowship at the same time?

Shannon:

At first, yes. And I mean this was little problems that he would come along with, like, like they had some smoothing things at Sperry, places like that.

Price:

And that’s where Preston Bassett came in I think.

Shannon:

Could be. Well, I remember they had a lot of fun, it brings back all these memories. They had some incredible complicated mechanical smoothing devices. Things that we would do with, you know, tiny . . .

Price:

Ball-disk integrators and such?

Shannon:

Yes, well, and more complicated, and yeah, many of those things.

Price:

Viscous dampers and . . . it’s a whole different world.

Shannon:

Yes. Well anyway, right, but anyway, he wanted me to help in the mathematical analysis of all this. And I did a little bit and I wrote a paper or two for his committee. And some of that may have been on that same —

Price:

And this is still about a year before Norbert Wiener's "Yellow Peril" comes out, right?

Shannon:

Yeah, I think so. I don’t know . . .

Price:

Because that came out in ’42, and here you were in late ’40, ’41.

Transition to Bell Labs; Shannon's personal papers

Shannon:

That’s right. But anyway, about halfway through this year, you know, things were moving fast there, and I could smell the, I could smell the war coming along. And it seemed to me I would be safer working full-time for the war effort, safer against the draft, which I didn’t exactly fancy. I was a frail man, as I am now.

Price:

Always nice to have an option.

Shannon:

Well, I was trying to play the game, to the best of my ability. But not only that, I thought I’d probably contribute a hell of a lot more.

Price:

Sure, well, Vannevar Bush was right and Roosevelt was right to set up the NDRC.

Shannon:

Yes, that’s right.

Price:

Whereas Hitler assumed he’s going to win the war hands down, he didn’t need that kind of fanciness.

Shannon:

So anyway, what I did, I resigned my Fellowship about half the way through. And I went down and asked Fry if I could get a job at Bell Labs.

Price:

And you left, when you resigned, you didn’t leave any terminal progress report behind at this . . . on what you had arrived at in communication theory?

Shannon:

I don’t think so.

Price:

Well, in your file at the National Research Council, there’s nothing but the proposal. And so unless you have it here in this house it doesn’t exist.

Shannon:

Yeah, I don’t think . . .

Mrs. Shannon:

If it’s here we haven’t found it yet.

Shannon:

I don’t think I left a summary of what was done.

Price:

By the way, you have me to thank if you want to thank, the Library of Congress requested that you donate your papers to the Library of Congress.

Shannon:

Oh, they did, they came through?

Price:

They want you to get organized sometime and . . .

Shannon:

I’m going to send you up to my attic. [laughter ]

Price:

They would very much like to have . . .

Shannon:

That would be a suitable punishment to fit your crime.

Price:

There’s the Vannevar Bush collection at the Library of Congress and they want to have Claude Shannon working papers.

Shannon:

Well, anyway, that’s how I got into the smoothing . . .

Mrs. Shannon:

That’s what you did in the war, Daddy.

Shannon:

That’s what I did, and I worked on that. And also I got . . . but let me put it this way, that I was not working on just one thing all this time. I was thinking about Information Theory at Princeton and at Bell Labs. All through this.

Norbert Wiener filtering theory; statistics

Price:

Let’s go back a second. I’d still like to go back to Wiener, because after all, he modeled the evasive action and the signal trajectory for the gunfire control problem, he modeled that as a stochastic process. That’s what his filtering theory was about. He called signal — I mean there was a noise process, which obviously is stochastic. And the signal process he modeled that as stochastic. Wasn’t he perhaps, the first therefore to identify a signal as a stochastic process, even though we might not call it a information signal but nonetheless, couldn’t . . .

Shannon:

Yeah, I think that some, a case could be made for that. And I give him all credit for this attempting to separate signal and noise by a linear filter, although it was . . . that is a very limited way of trying to do it.

Price:

Sure, but Kolmogorov had done the same thing.

Shannon:

That’s right, Kolmogorov and previous to Wiener.

Price:

Yeah, it was a good model, so maybe they both could be credited with having a meaningful signal being a stochastic process.

Shannon:

See, the step forward here, separating wanted from unwanted is an old game in statistics. The step forward made in this case was that it was an infinite continuous process rather than a finite dimensional space. If you go into statistics you’ll find that everybody had been concerned with this problem of separating signal from noise because what are statisticians doing, they’re reading noisy data all the time. They’re trying to get the truth underlying it.

Price:

But did they ever model the data as a stochastic process?

Shannon:

Well, it’s a statistical process.

Price:

To me maybe, but to put a priori probabilities on it . . .

Shannon:

Oh yeah, of course there were. Oh yes. That was . . .

Price:

That did not originate with Wiener.

Shannon:

Well, the difference, Bob, is whether there is an infinite-dimensional continuum thing, or a finite space. The problems you’ll find in the statistics book are making the best estimate of something. They’re called an estimation problem or something like that. The best estimate of something there, looking for the signal there, when they’re given noisy data, when they’re given a bunch of readings, you see, of some particular measured readings of how long this object is, with all sorts of errors in. On all these readings they try to get the truth underlying it, and separating that signal from that noise.

Price:

Regression analysis.

Shannon:

Right. All those things, and they’re, the same thing, the difference here is applying it to a continuing infinite dimensional process if you like, of a wave that goes on forever.

Price:

Whether it’s discrete-parameter or continuous-parameter.

Shannon:

Well, I mean they’re both —

Price:

They’re both generalizations.

Shannon:

And I find that generalization interesting but they’re not —

Information theory and Norbert Wiener

Price:

So even though you studied Wiener’s "Yellow Peril" intensively where the signal was modeled as a stochastic process and therefore could be thought of, information being modeled as a stochastic process, that didn’t trigger anything with you because you already had envisioned information as a stochastic process, before the Yellow Peril.

Shannon:

Yes.

Price:

Before the Yellow Peril, that’s my question? Or did it perhaps provide the missing link that you had been looking for. Peter Elias suggested that you started out in Information Theory in 1940 with the sphere-packing idea, not stochastic but sphere packing.

Shannon:

Well, I had some of those ideas, I don’t think I had that sphere-packing picture . . .

Price:

He felt you might have had a counting argument, as he puts it, and the information at that point didn’t happen to be stochastic.

Shannon:

Did you read Hartley’s paper?

Price:

No, I never did really study it, but I know he got a [Institute of Radio Engineers] Medal of Honor for it.

Shannon:

He was far more a stepping stone for me than was Norbert Wiener.

Price:

Because of the logarithmic measure?

Shannon:

Well, because he was interested in communicating information.

Price:

Yes. Well, Wiener eventually was, but not early, not in your time frame.

Shannon:

Well, yeah, well, I don’t —

Price:

Communication and Control in the Human and Animal . . . Human and Animal Machines . . .

Shannon:

Well, that’s just a name of a book. It’s not at a deep level. You know I don’t believe that Norbert ever understood what I did.

Price:

I believe that his son-in-law feels the same way. [laughter] Toby Raisbeck. Well he, Toby, wrote this letter that said . . . that Norbert Wiener thought he knew what you did and therefore he never studied what you did.

Shannon:

I think, I think that may be true. But when I talked to Norbert, like in the 1950s and so on, I never got the feeling that he understood what I was talking about.

Price:

Well, I’m sure one thing he never knew was the Second Fundamental Theorem, you know. That was too, that was too much of a surprise — anybody who assumed that they knew what you did would never have imagined the Second Fundamental Theorem — because it’s so counterintuitive. It’s like E =MC2. Therefore he never interested himself to find out what was truly phenomenal in your work there.

Shannon:

Well, Norbert Wiener is a very interesting man, in all ways. I mean I’ve read his books, his autobiographical books and his scientific books.

Price:

They’re very enjoyable to read.

Shannon:

And I’ve talked to him much and I’ve listened to him lecture. I still must say I don’t understand . . I never did understand him. [laughs]

Price:

He made a point maybe of making sure that people couldn’t really understand perfectly.

Shannon:

I think he had a great brilliance. I’m not putting down his great mind. I think he really did have a tremendous IQ and a tremendous grasp of many things.

Information processes as stochastic, probabilistic

Price:

Yeah. Well, can we leave, is the question to be left sort of uncertain yet as to when you started modeling information processes as stochastic or a probabilistic process?

Shannon:

I would say 1941.

Price:

And that’s clearly ahead of the Yellow Peril, 1941. But did it occur to you because of some Bell Labs work? You weren’t into cryptography yet in ’41, were you?

Shannon:

No.

Price:

So, can you remember any context in which it occurred to you that if you knew the probability to be either 1 or zero ahead of time, there was no information therefore? And if that was the missing . . . well, Hartley, you say, but Hartley, that doesn’t have to be probabilistic on the signal, only on the noise. He just brought, divided it up into cells by the noise statistics, and he didn’t do that, but that would have been fairly straightforward and fairly pedestrian. To model the signal as probabilistic, that to me is the conceptual breakthrough.

Shannon:

I don’t regard it as so difficult. I mean to me one would think of what would be the simplest source you might have, or the simplest thing you were trying to send. And I’d think of tossing a coin, heads or tails, and to try to send that stream of data. And that would seem to me, that did seem to me, as a way to start, so to speak. And what do you need to do that? And is it any easier if the coin is loaded? And if it has six sides like a die, and so on?

Price:

Well, things like the penny-weighing problem, they weren’t that early were they?

Shannon:

The penny-weighing problem, I don’t know when that, I think they might have been that early but I don’t know that that would influence me.

Price:

No, but it could lead you in a probability direction, couldn’t it?

Shannon:

Yes — I don’t, that’s an interesting question: how many tests, how many weighings do you need to find a false coin?

Price:

Yeah, Fano, who taught me, had independently arrived at that as a useful example.

Shannon:

Yeah, we haven’t mentioned Bob Fano, but he was in this game quite early too. And he was, he came down and talked to me at Bell Labs at various times about all this.

Price:

Right, but according to the M.I.T. EE & CS Department’s “History” [A Century of Electrical Engineering and Computer Science at MIT, 1882-1982] that Karl Wildes is writing, Fano got his cue from Wiener. That he was looking for something to do after he had done a lot of electromagnetics theory during the war. And he was stimulated by Wiener’s modeling of this filtering problem. And I think it actually says in the “History” that he was also stimulated by Wiener’s suggesting that information was a probabilistic process. So though you didn’t get it from Wiener, it could still be that he was circulating that around.

Thermodynamics and entropy; cryptography

Shannon:

Well, let me also throw into this pot, Szilard, the physicist. And von Neumann, and I’m trying to remember the story. Do you know the story I’m trying to remember?

Price:

Well, there are a couple of stories. There’s the one that Myron Tribus says that von Neumann gave you the word entropy, saying to use it because nobody, you’d win every time because nobody would understand what it was.

Shannon:

[laughs]

Price:

And furthermore, it fitted p*log(p) perfectly. But that, but then I’ve heard . . .

Shannon:

von Neumann told that to me?

Price:

That’s what you told Tribus that von Neumann told that to you.

Shannon:

[laughs – both talking at once]

Price:

Bell Labs too, that entropy could be used. That you already made that identification. And furthermore in your cryptography report in 1945, you actually point out, you say the word entropy exactly once in that report. Now this is 1945, and you liken it to Statistical Mechanics. And I don’t believe you were in contact with von Neumann in 1945, were you? So it doesn’t sound to me as though von Neumann told you entropy.

Shannon:

No, I don’t think he did.

Price:

This is what Tribus quoted.

Shannon:

Yeah, I think this conversation, it’s a very odd thing that this same story that you just told me was told to me at Norwich in England. A fellow —

Price:

About von Neumann, you mean?

Shannon:

Yeah, von Neumann and me, this conversation, this man, a physicist there, and I’ve forgotten his name, but he came and asked me whether von Neumann, just about the thing that you told me, that Tribus just told you, about this fellow. . .

Price:

That was Jaynes, I imagine the physicist might have been [Edwin] Jaynes.

Shannon:

Yes, I think it was, I think so. Do you know him?

Price:

Well, he’s published in the same book as Tribus, you see. This is a book called The Maximum Entropy Formalism. You’ve probably seen that book, but they have chapters in it, and Jaynes, the physicist —

Shannon:

Now, I’m not sure where I got that idea, but I think I, somebody had told me that. But anyway, I think I can, I’m quite sure that it didn’t happen between von Neumann and me.

Price:

Right. Well, I think that the fact that it’s in your 1945 cryptography report establishes that, well, you didn’t get it from von Neumann, that you had made the p*log(p) identification with entropy by some other means. But you hadn’t been —

Shannon:

Well, that’s an old thing anyway, you know.

Price:

You knew it from thermodynamics.

Shannon:

Oh, yes, from thermodynamics. That goes way back.

Price:

That was part of your regular undergraduate and graduate education of thermodynamics and the entropy?

Shannon:

Well, not in class, exactly, but I read a lot, you know.

Cryptography report and publication

Price:

Well, fine. Well, your 1945 cryptography report is very interesting because on the one hand for the binary case you say “alternative,” you don’t say “bit,” you use the word alternative. The decision between zero and one is an “alternative,” so that clearly took Tukey a year later to bring it to “bit.” On the other hand you use the word entropy and in one place you actually say “Information Theory,” and this is in 1945. You just make a passing reference to Information Theory. And I think that’s the first time that phrase has appeared, or those two words have appeared together in print anywhere.

Shannon:

Is that so. Well, that could be. You know, that’s a funny thing that cryptography report, because it contains a lot of Information Theory which I had worked out before, during those five years between say 1940 and 1945. And those are the war years. And things that I did around the time at home, not office hours, so to speak.

Price:

So some of that Fellowship work is actually in that cryptography report?

Shannon:

I would think, yes, some would be and I believe I constructed random sentences and stuff, you know, or random sequences of letters, which approach as you get more and more distant statistics you get more and more like English, one thing or the other.

Price:

You did that Markoff analysis of letter strings during the war?

Shannon:

Yes, that was all, that was all in that report, I think.

Price:

Wasn’t that motivated by cryptography?

Shannon:

That was, my first getting at that was Information Theory. And the cryptography, I used that as a way of legitimatizing it all, you know [laughs] if you understand what I mean. To make it sound like I’m working on decent things. [laughs]

Price:

Was it an answer looking for a problem then, sort of? You were delighted to find cryptography coming along during the war, something needed [CS laughs], and that was a great application of your Information Theory?

Shannon:

In fact. You might say that cryptography was there, and people were working on secrecy systems in Bell Labs, and it seemed to me that here this was very closely related. And I should go over to that too. And the other thing I was not yet ready to write up the Information Theory anyway. This you could write up anything in any shape, which I did.

Price:

Well, it’s very interesting, very well documented. As you know, it was censored when it was published in 1949 as your Communication Theory of Secrecy Systems, great chunks of it were left out, even though you said it was declassified, great chunks were left out.

Shannon:

Well, I may have cut them out myself.

Price:

Well, about the modified, the Compound Modified Vigenere cipher?

Shannon:

Yeah.

Price:

Yeah, that was censored out.

Shannon:

Well, I don’t, I don’t remember any of those details.

Price:

Sure. Well now, I believe Peter Elias gave you a copy of this cryptography report, which you’d never had before, didn’t he? Did he give you a bound copy of your cryptography report which was was classified for many years, so you couldn’t take it with you. But when he visited you in the past year didn’t he give you a copy of the cryptography report?

Shannon:

Yes, I guess he did. I think I, for some reason never got, I never looked it over.

Mrs. Shannon:

We don’t know where it is if we have it.

Shannon:

Yeah, I kind of have a vague recollection but I don’t remember studying it in detail.

Price:

So what you’re telling me about it is from your recollection? I thought you had looked at it again more recently.

Shannon:

I haven’t looked at the . . . nor the paper either since 1949. [laughs]

Price:

Well, do you think that even if there hadn’t been a war effort that you would have been interested in the cryptographic aspects of this?

Shannon:

I might. I probably would. That kind of thing is the thing that attracts me. But I mean I was a great fan of Edgar Allan Poe and "The Gold Bug" and stuff like that.

Price:

Yes, I know. That’s a great story. I wish they would make a film of "The Gold Bug."

Shannon:

Yes, and I used to solve cryptograms as a boy.

Price:

I used to too. I remember Edgar Allan Poe said that any cryptogram a man can create a man can decipher. I think he offered, he would pay somebody if he couldn’t decipher their cryptogram.

Shannon:

That’s right.

Relationships between cryptography and information theory

Price:

So, but you see, I’ve read, for instance, John R. Pierce says that cryptography was an application of Information Theory. I was pretty sure that was putting the cart before the horse. But I was beginning to think that it was the other way around, even that Information Theory had come out of cryptography, your cryptography study, because here I’m looking at this 1945 cryptography report, it’s got the words Information Theory in it. It says you’re next going to get around to write up Information Theory. It sounds . . .

Shannon:

Oh did it say that in there?

Price:

Yes. It sounds as though the cryptography gave you the mysterious link that made the whole, well, the fan diagrams, for instance, if it hadn’t been for cryptography would you have had those fan diagrams?

Shannon:

What fan diagrams?

Price:

Well, the, you know, the e-to-the-entropy, going that way, and the most probable received signal set for a given transmitted — it’s sort of the analogy of the sphere-hardening business, but the fan diagrams . . . Well . . .

Mrs. Shannon:

This is marvelous. [laughter]

Price:

Well, maybe we just called them fan diagrams as graduate students. But you —

Mrs. Shannon:

He never had that course as a graduate student, you see. [laughs]

Price:

It’s the code, it’s the second, it’s how you proved the Second Fundamental Theorem.

Shannon:

Well, these things?

Price:

These. These. [pointing to the fan diagrams on page 41 of The Mathematical Theory of Communication, 1949].

Shannon:

Oh, those are fan diagrams. [laughing]

Price:

Well, do you think that would have come out of cryptography? Or you had that already without the cryptography?

Shannon:

Well, I have no idea.

Price:

OK, sure, well it’s a long time ago.

Shannon:

But not only that, Bob, you ask questions of where would things have come from? These complex hypothetical questions.

Price:

Sure. OK, but anyhow, it’s clear that Information Theory did not come out of cryptography.

Shannon:

No.

Price:

On the other hand, cryptography wasn’t just strictly . . .

Shannon:

Audio File

MP3 Audio
(423 - shannon - clip 2.mp3)

Cryptography was a great . . . working on that led back to good things in Information Theory. I started with Information Theory and the real thing for me was Hartley’s paper which I thought was a good paper, but I thought had not yet taken account of things like noise and best encoding and stuff like that. They hadn’t really done a maximum-minimum, [laughs] perhaps min-max. So I, and incidentally, Hartley was still working at Bell Labs when I went there, and I talked to him several times. He was kind of an unusual character. He was, he didn’t like Einstein’s stuff, and he spent the latter part of his life trying to get back into classical physics. But anyway, going, continuing what I was talking about, what was the main question I was addressing?

Price:

Well, cryptography vis-a-vis Information Theory. And I think I read from the Hagemeyer interview [cited in the Doctoral thesis by F.-W. Hagemeyer, “Die Entstehung von Informationskonzepten in der Nachrichtentecknik,” Free Univ. of Berlin, 1979] and so forth, and you’ve said it to other people, that these were really intimately intertwined, and that cryptography was no mere application of Information Theory. You say you got stimulus. Could I suggest there was sort of a duality there? I’ve noticed some things that the cryptography case was in some example, in some cases, some ways the mirror image of the communications problem. That’s why maybe you got some new insights out of it.

Shannon:

Yeah, I think I’ve made some remarks like that in one of those papers.

Price:

OK. Well, anyhow as I say, John Pierce in his book, Signals, Symbols and Noise, I think had the wrong impression that cryptography was just an application of Information Theory, whereas it was a stimulus to it, in effect, right?

Shannon:

Yes, well, I think that all these sciences and theories, you know, stimulate each other, if they’re related at all. In my case I started with Hartley’s paper and worked at least two or three years on this, as information, the problems of information and communication. And that would be around 1943-44, and then I started thinking about cryptography and secrecy systems. And I wrote this paper, which in a sense sort of justified some of the time I’d been putting into this other subject, at least in my mind. [laughs] And so that, but there was this close connection. I mean they are very similar things.

Price:

They certainly deal both with information.

Shannon:

Information, at one time trying to conceal it, and at the other time trying to transmit it.

Price:

So there’s a duality there.

Shannon:

Yes.

Price:

Both entropy measures can be used in both cases.

Shannon:

Yes. And when I came out with my paper in ’48, part of that was taken verbatim from this secrecy thing, with this cryptography paper, which had not been published as yet. And as a matter of fact I did a kind of a tricky jigsaw job of using part of it in the Information Theory and then later part of it in the secrecy, in the cryptography paper. You may have noticed this. You’ve been doing such a deep job here, you probably realize that.

Multidimensional geometry, sphere-hardening

Price:

Then there’s a third paper, extremely important, which you published in the Proceedings of the IRE, in which you used the sphere hardening and the Somerville multidimensional space stuff. And the curious thing about that paper is, in some respects it’s a sequel to your BSTJ great opus here. In other respects it’s a different approach, because you don’t have the multi-dimensional geometry in this Shannon and Weaver book at all. Peter Elias has suggested that that was the direction you were starting on earliest, the paper you actually published later in ’49, in January of ’49, that you had been starting on that at the Institute for Advanced Study, about 1940. That multidimensional take-off was a logical thing building up partly by a counting argument.

Shannon:

Well, I can’t even answer that. There’s some, I can see some possibility of that, because my thinking then was in the continuum, and the Heisenberg Uncertainty Principle, which relates very closely to the number of dimensions. I remember talking to Weyl, Hermann Weyl about this business of the dimensionality as you increase the time, available time. If you, it’s sort of a one-dimensional thing, if you get into the tightest Heisenberg uncertainty thing, and then if you go farther, you get two-dimensional, more and more time, or more and more bandwidth and so on.

Price:

Did Weyl perhaps point out that you got a sort of sphere-hardening effect as you went to multi-dimension?

Shannon:

I don’t remember that.

Price:

This was in Somerville’s book, well, you interpreted it that way . . . asymptotic perfect reliability.

Shannon:

Somerville, what’s the name of his book?

Price:

D. Y. N. Somerville.

Shannon:

What’s the name of the book, do you know?

Price:

Multidimensional Geometry [actually,The Geometry of N Dimensions], something like that. You thought it was a very valuable book at the time. You had discovered it and it provided [CS laughs], sort of like later on . . . like the Chernoff Bound, later on you found that useful, you found Somerville’s book useful early on for the content of spheres, gaussian spheres, gaussian noise.

Shannon:

Isn’t that funny, I can’t remember Somerville’s book.

Price:

I don’t have it right here. It became a Dover reprint later on. It was called The Geometry of N Dimensions, and it’s a British book.

Shannon:

Well, I’ve read many books on N-dimensional geometry, but Somerville, the name rings a bell, but I don’t get it this connection.

Price:

It’s spelled just like Somerville.

Shannon:

Anyway, we have it right in the next room, so I know we have it.

Price:

Well, you highlighted that book in your general 1949 paper, and we all tried to go out and get it, and it was out of print, it had been out of print for 40 years. But you . . . then Dover reprinted it.

Mrs. Shannon:

He made it a classic.

Shannon:

You should be a D.A. [laughs] You can gather all the facts and put it right in a guy’s face.

Price:

Attempted facts. The . . . OK, now there’s a very . . .

Herman Weyl; gaussian noise

Shannon:

Well, that may well be. I was kind of a nut on high dimensional geometry in college. I mean undergraduate college, and things like that.

Price:

You went to the right person. You went to Herman Weyl for . . .

Shannon:

Yes, he was a good man.

Price:

Now . . .

Shannon:

What was I going to say. He . . . I remember the very first lecture I went to where Herman Weyl talked about, he gave, at Princeton. And it very nice. Now, it had to do with how many things you could pack into a certain space, proving that — like how many spheres could you pack into another sphere, something of that sort, in a dimension, or whatever. Which is of course very closely related to all this subject. The bounds were often crude and they were based on, the spheres had to not intersect. But anyway it was oddly related to the sort of things I’d been thinking about, although in an entirely — this was just a pure geometrical problem of geometry, of wondering how many spheres can you get of a certain size inside of another sphere.

Price:

Well, you were thinking of gaussian noise.

Shannon:

Yeah, stuff like that.

Price:

But at least it was radially symmetric gaussian noise.

Shannon:

Yes, and my things could intersect some, but I didn’t want too much overlap, you see.

Price:

You had a hunch, you said you didn’t want too much overlap. But now this would, I mean how could you imagine that there couldn’t be overlap asymptotically. What gave you the idea that there could be asymptotically high reliability at a finite rate? That’s counterintuitive.

Shannon:

Yeah, well, in high dimensions you’ve got to get a feeling for high dimensional space.

Price:

But as a communications engineer why would you think that you could ever defeat noise in the long run to get perfect reliability without just repeating yourself over and over again? I mean that’s, I’m talking about the Second Fundamental Theorem, which is, which I must say, is the greatest contribution it seems to me, because you know, that’s like E = MC2, a special case being log(1+S/N), but the second, the coding theorem for the noisy channel is . . . you know, for a long time there were few . . .

Shannon:

Well, I appreciate those kind remarks. I think you overestimate the beauty and value of the whole thing, but —

Price:

Well, it’s a conceptual thing. It provides a baseline, but the fact that it even was possible to get asymptotic reliability — you didn’t know that in the beginning. You didn’t know that in 1940.

Shannon:

No. No, I don’t know when I first realized it.

Price:

Well, people were worried, you know. It wasn’t until [Amiel] Feinstein came along that people could absolutely be sure by independent means that your theorems were valid. You know what Doob said.

Shannon:

No, well, which, what did he say?

Price:

In the review of your . . . intentions? Are your intentions honorable?

Shannon:

Yes.

Price:

Well, that’s what the mathematicians were worried about. And then Bob Fano, when he got Feinstein to prove the way he wanted, then he could breathe a sigh of relief. He was hoping it was true, but only you knew that it was true apparently.

Shannon:

Well, I’m glad they agree now.

Frequency model publication; 1939 letter to Vannevar Bush

Price:

Oh, it’s just great. Lemme, now there’s a mysterious question about this 1949 paper, because it has a submission date of 1940.

Shannon:

Which paper?

Price:

This is the paper published in the Proceedings [of the IRE] in 1949, in which you for instance model FM as the wiggly line, the mapping of the one dimension into two, and that’s why you have break points in FM and noise capture. Remember the wiggly line mapped into the square? That was a model, a frequency model.

Shannon:

Yes, yes, I remember the wiggly line. [laughs]

Price:

And you had the water pouring, what we called the water pouring of how to distribute the transmitted power if you have a non-white noise in the channel. And your total amount of transmitted power, you treat it like water and pour it around uniformly in the valleys, which wasn’t in this 1949 book [The Mathematical Theory of Communication] at all. In fact, in this book you said that was an unsolved problem but you solved it some months later and published it. [in “Communication in the Presence of Noise,” Proc. IRE, vol. 37, pp. 10-21, 1949] Well, it has this multidimensional geometric thing. That’s the most distinctive thing, perhaps to some, you know, with a Somerville reference. And it gives a submission date of 1940, first submission, 1940.

Shannon:

Well, that is an error of some sort.

Price:

Well, we were wondering if maybe you hadn’t submitted something tentative in 1940 when you thought you were on the track of some improvement of Hartley and decided to withdraw it and refine it. But the IEEE archives has no record of this. It looks as though it could maybe be a misprint for 1946, you know, the 6 might have been scribbled as a 0 accidentally.

Shannon:

I don’t think there’s any error at all [sic]. I think the date given, that is, that would be a year like a year before publication at most.

Mrs. Shannon:

That was probably typed at Bell Laboratories and chances are the file . .

Shannon:

There certainly was no vast disagreement. And I had no paper like that in 1940.

Price:

You had nothing to submit, even in the way of a note in 1940?

Mrs. Shannon:

You were up here at M.I.T. in 1940.

Shannon:

In 1940, I graduated from M.I.T., yes.

Price:

Well, yes, but you wrote a letter just to Vannevar Bush, it’s in the Hagemeyer thesis, explaining what your intentions were going to be in studying information theory.

Shannon:

In 1940?

Price:

Yes, if you look in the back of the German thesis you’ll see your letter to Vannevar Bush.

Shannon:

Oh, I’d like to see that.

Price:

And it gives the intentions. And I thought you might have carried them far enough along that you actually would have written a preliminary note to theProceedings of the IRE, and then later withdrawn it, and after the war when you were satisfied with the state of things, then submitted it for full publication. You don’t remember submitting anything to a journal in the line of communication or Information Theory, as against switching theory or genetics?

Shannon:

Well, I, well, that is very —

Price:

It’s 42 years.

Mrs. Shannon:

I don’t know where it is. I haven’t found it yet.

Shannon:

Well, let me look for it. I think I’ll recognize it.

Price:

It’s right toward the back [of Hagemeyer’s thesis]. It’s almost an Appendix.

Shannon:

Oh, I’ve just . . .

Price:

That’s it. Isn’t it? Is that to Vannevar Bush?

Mrs. Shannon:

“Dear Dr. Bush. . . .”

Price:

Yes.

Mrs. Shannon:

Yeah, I guess it is. I was going to say, if that was a paper that was published in IRE in ’48 or ’49 . . .

Price:

It was published in ’49.

Mrs. Shannon:

It would have been typed at Bell Laboratories, and it’s probably in their files.

Price:

Well, yes, I don’t think I actually put that question to them in quite that way. I put it to the IEEE, and there are some people there who have been there since 1940 and they would remember a paper that was withdrawn and republished nine years later, even though you don’t have a copy of it.

Mrs. Shannon:

That doesn’t ring true.

Price:

That doesn’t seem logical.

Mrs. Shannon:

No, it doesn’t seem reasonable.

Price:

So it seems like it was a misprint.

Mrs. Shannon:

And it may even have been circulated at Bell Laboratories as an internal monograph with a routing list on it, and you might find a copy of that sitting around in their files too.

Price:

Yes, that seems like a logical way to answer that. But the probabilities, it’s virtually impossible that anything would have been submitted in 1940.

Mrs. Shannon:

Oh, it doesn’t sound reasonable at all.

Price:

But there’s the letter to Vannevar Bush and that’s dated 1940, but that’s the . . .

Shannon:

Well, that —

Price:

That’s the intention to do the work you’re going to do, right?

Shannon:

Exactly so. Well, this is very interesting.

Price:

You haven’t seen that for a long time.

Mrs. Shannon:

Where’s the date on it?

Shannon:

Yeah, where’s the date here?

Mrs. Shannon:

Oh, here it is, December, ’39.

Shannon:

’39. [laughs]

Price:

There you are. OK, ’39, so, see, that’s what I say, could you have submitted something in ’40?

Mrs. Shannon:

It doesn’t seem likely.

Shannon:

Well, this is, this is great stuff. [laughs]

Price:

A self-fulfilling prophecy.

Shannon:

You think of this smart-aleck kid trying to do this when he’s hardly out of college, right.

Price:

Hagemeyer didn’t tell you about that, apparently. See, he got that out of the Library of Congress in the Bush Collection in Washington. So ’39, that’s 43 years ago.

Shannon:

See here, “roughly speaking, it is impossible to reduce bandwidth times transmission time for a given distortion.”

Price:

That’s the Indeterminacy Principle.

Shannon:

Yeah, that sums it up very well, here.

Genetics work, 1939-1940

Price:

I think this is a reply of yours to Vannevar Bush when he was trying to persuade you to go into genetics, you were telling him sort of that you wanted to do this instead.

Shannon:

Well, it’s more than that. See, there was another summer here, maybe I’m mixed up on the summers. But anyway, Van Bush was the head of the Carnegie Institute at this time or something like that.

Price:

Institution.

Shannon:

Institution, yes. And one of their things there’s a genetics place down in Cold Spring Harbor, New York. And he suggested I might work down there one summer. Now what the hell summer was that?

Price:

That’s on the adjacent page. That’s also in the Bush Collection.

Mrs. Shannon:

That might have been ’39-’40.

Price:

There’s a letter from Barbara Berks, do you remember her?

Shannon:

Oh, how could I forget her. Barbara Berks, I see it, yes.

Mrs. Shannon:

That may have been ’39-’40, and then ’40-’41 you were down at Bell Labs. That’s probably . . .

Shannon:

My goodness, she did me good, darling, read it. She says, “when one rereads Shannon’s paper, the impression of clarity of thought and imaginative scope is, if possible, intensified. The only parallel . . . ” [laughs] this is like reading a theater review.

Mrs. Shannon:

Rave on, rave on.

Shannon:

If you’re a great actor or something. “To advise a youth like Shannon, is difficult, is it not?”

Mrs. Shannon:

It’s hard to think of you as a youth somehow. [laughs]

Shannon:

It’s too perfect.

Mrs. Shannon:

This was ten years before I knew him.

Price:

She wanted you to continue in genetics obviously.

Shannon:

Well, not . . . yes. Now it comes clear to me. This was the summer of . . . it sounds like that movie, doesn’t it. From ’43 . . . but this was the summer of 1939.

Mrs. Shannon:

’39-’40, yeah, it would be ’39.

Shannon:

Yeah, because this is where I worked on the genetics for the Carnegie Institute, and then used some of that work in my doctoral thesis the following year. So all this is the way that goes.

Price:

Yes, then the war came along and changed things in a way. But if the war hadn’t come, well, you can always say, “what if?”

Mrs. Shannon:

Yeah, who knows?

Price:

Information Theory would have been retarded, if the war hadn’t come.

Mrs. Shannon:

It might never have — yeah.

Shannon:

Might never have been, who knows. [laughs] Or I might have fallen among bad companions.

Mrs. Shannon:

Never at Princeton.

Shannon:

Not at Princeton. No. But anyway, this is interesting. Well, especially this one about the Bush correspondence

Transition to communication theory, information theory, and cryptography

Price:

Yes, your plans for doing a proper synthesis of communication theory.

Shannon:

Yes.

Price:

By the way, I have the cryptographic paper here.

Shannon:

What exactly are your plans, Bob, and what is the reason for . . .

Mrs. Shannon:

What are you going to do with all this miserable stuff?

Shannon:

Why should you undertake such a ridiculous project?

Price:

Actually, I’m not, I’m interested in the roots of Information Theory, of course, but I’m not undertaking to redo what Hagemeyer has already done. In fact, I think a lot of this must be going over old grounds, because you’ve already done this four years ago into a tape recorder. And when the Hagemeyer thesis eventually can get translated, you’ll see it all there, probably a lot of duplication of what you’re telling me right now. I’ve sort of come into Information Theory here by the back door, because I’ve been really working on a history of spread-spectrum communications. But this was developing about the same time that your information theory publications came out, or actually afterwards. And I was looking for links between the two. And when I was working with Bill Davenport and Paul Green at Lincoln Lab in the early days using noise-carrier signals, we would sometimes — it was a Secret project. We’d sometimes say, gee, wouldn’t Dr. Shannon at Bell Labs be interested to know about this, even though it’s not really Information Theory. It’s signaling with gaussian noise for military reasons and there’s overlap areas here, so it would be sort of interesting to tell him about it and see what his reaction would be. We never did, because we weren’t sure that you had any clearance, and your theories weren’t directly applicable. The only thing we had in common is we were both conceiving the use of noise-like signals. So now 30 years later we’re writing the history of . . . spread spectrum …

Price:

Solomon Kullback was a statistically oriented cryptographer, or is. I mean, he’s retired in Florida. He’s one of three people hand-picked by William Friedman to do cryptography work on the basis of statistics. And they say Friedman is the father of statistics in cryptography. And his wife was also a cryptographer. I’m sure you all know about that. Because you have The Codebreakers. It has your photograph in it, and Kullback’s photograph is in the Codebreakers book too, by David Kahn. The great opus on cryptography.

Mrs. Shannon:

I don’t think we have it. We have the chapter of it that he sent us. I don’t think we have the whole book.

Shannon:

I didn’t know my picture was in it.

Price:

Oh yeah, and Kullback and . . . so anyhow . . .

Shannon:

This is, we should have our own tape recorder on. We’ve got all this good information coming this way. You think you’re getting the information.

Price:

Well, you must have been —

National Security Agency; WWII and cryptography intelligence

Shannon:

Kullback, as you say is, more and more, the name rings more and more bells.

Mrs. Shannon:

I’m sure that it was his son who baby-sat for us out in California and that the name came up then.

Shannon:

Well, that’s —

Price:

Well, I’ve spoken to him several times on the phone and he —

Shannon:

He was in — was it mainly his life in cryptography?

Price:

Well, until after the war. He was an officer in the Army Security Agency during the war working with Friedman, in what became the NSA after the war. And you were on the board of the NSA, weren’t you for a while?

Shannon:

I don’t think I was on the board. I might have been a member. I don’t think I was that, that elevated a position.

Price:

Well, you had dealings with the National Security Agency at some time I’ve been told.

Shannon:

Yes, that’s a better way to put it.

Price:

And that was after, not during — the NSA didn’t exist during the war. But this is something else, Dr. Bennett, William Bennett . . .

Shannon:

Bill Bennett, yes I know him.

Price:

and today, Professor Tukey, I spoke to one yesterday, and one today, they both said that you did your cryptography work during the war with, it wasn’t without authorization but you had absolutely no need to know. You were never told anything during the war. You did the cryptography but you were never privy to what was really going on in cryptography.

Shannon:

That’s a pretty accurate statement. I had no, in fact very little interest in what was going on. That’s well put.

Price:

Well, you knew, you apparently knew about important speech scrambling work going on at the Labs, at Bell Labs, that helped to get you even more motivated that you might make a contribution in cryptography.

Shannon:

Yes, I knew of the existence of it. And I knew how stupid much of it was.

Price:

Do you know what [R. C.] Mathes and [R. K.] Potter were trying to do, for example?

Shannon:

Those names, yeah, I know both those guys.

Price:

They built the system that protected the Roosevelt-Churchill telephone conversations?

Shannon:

Well, I didn’t know that particular fact.

Price:

You weren’t supposed to know.

Shannon:

I knew Ralph Potter.

Price:

Yes.

Shannon:

Yes, I know that there was quite a group working on speech, what do they call it?

Price:

Encipherment or scrambling.

Shannon:

Scrambling, speech scrambling.

Price:

To do it better because the ordinary band permutation thing had failed, the old fashioned way of privacy speech, this was moving the bands around. They could easily be deciphered.

Shannon:

Not only easily. You could just listen to it and a clever person could understand it.

Price:

Just psychologically. And then Hitler . . .

Shannon:

It’s like a person talking with a kind of a weird accent, almost.

Price:

Right. So it came out later on that in fact, Hitler had the messages delivered to his desk shortly after Roosevelt and Churchill . . .

Shannon:

In many cases, I think so.

Price:

But then the new system came in called the Green Hornet, or X-System, which Potter and Mathes worked on. But you weren’t supposed to know anything about that, according to what I’ve been told.

Shannon:

We used to go to the movies with Potter.

Mrs. Shannon:

Yes.

Shannon:

But he never told us about that. Just a private joke [laughter]

Price:

Well, what Professor Tukey said, who was there at the time, I believe, he was there even during the war, wasn’t he?

Shannon:

Yes.

Price:

He said that all the information exchange would have been one-way between you and your visitors. It would have been from you going out, but your visitors in this bailiwick of cryptography, like Kullback, for instance, would never have said anything, they would just listen.

Shannon:

I think that’s pretty much true.

Alan Turing: cryptography, speech, and computers

Price:

It’s fascinating. Now, here’s the real question then. It is known that you had a distinguished British visitor during World War II.

Shannon:

Yes.

Price:

And I. J. Good knows what he had, he and Turing, we’re talking about Turing, of course, they were over in England deciphering German cryptographic traffic. And Turing came over to visit the United States. That’s well documented. He was at West Street for six months, I’m told.

Shannon:

Yes. The reason I’m hesitating, I’m trying to decide whether it was West Street or Murray Hill. And I think it was West Street, and I think, at least I knew of him there about six months.

Price:

Yes, it said that you saw a good deal of him when he was there?

Shannon:

Not a huge amount, but quite a bit.

Price:

Well, his business was not supposed to be known, what he was doing over here, because what he was doing has only become non-secret since 1974. It was held under secrecy for 30 years because he had succeeded in deciphering the German traffic. And if word of that ever got back to the Germans they could have changed their codes and ruined everything. And Churchill died, never being able to say anything about this. All he would say is, “this is my most valuable source.” That’s all Churchill ever said. But it was the ability using high-speed electronic equipment in England, to read the German cryptograms very rapidly after they went out. And these were the cryptograms generated by the Enigma machine.

Now I thought that, you see, earlier I thought that you had the clearance and need to know because you did this cryptography report, that you and Turing were comparing notes. But according to Professor Tukey and to Dr. Bennett, no, they couldn’t, Turing wouldn’t have said anything of that sort about cryptography, or at least the nitty gritty of what they were doing.

Shannon:

Certainly not the nitty gritty. I knew or surmised what he was doing.

Price:

You surmised.

Shannon:

See, I knew he was associated with, see, there was another guy John Riordan, and I don’t know whether you’ve run across —

Price:

The combinatorial expert.

Shannon:

Yes.

Price:

Who was also, had clearance in this cryptography area.

Shannon:

That’s right. And he was kind of involved in that. He was a good friend of mine. Andrews, did you mention Andrews, yes?

Price:

I don’t think I did.

Shannon:

Audio File

MP3 Audio
(423 - shannon - clip 3.mp3)

Andrews. Well, there were a number of people who were known to be, to me, working in speech encipherment and so on. Although I wasn’t working in it myself. And I would, and I wasn’t spending time professionally with Turing, but I’d often have lunch with him or something like that. Often, John Riordan and I would have lunch together and Turing and I.

Price:

I see.

Shannon:

And we would talk about mathematical subjects. Turing led--perspective here: Turing of course is most famous for the notion of the Turing Machine, and the related concepts, which was a very driving motive in my thinking all these years, although we haven’t touched on it here at all. The notion of building computers that will think and what you can do with computers and all that. And so, Turing and I had an awful lot in common, and we would talk about that kind of question. He had already written his famous paper about Turing Machines, so called, as they call them now, Turing Machines. They didn’t call them that then. And we spent much time discussing the concepts of what’s in the human brain. What the, how the brain is built, how it works and what can be done with machines and whether you can do anything with machines that you can do with the human brain and so on. And that kind of thing. And I had talked to him several times about my notions on Information Theory, I know, and he was interested in those.

Price:

Can you remember any feedback he might have given you on that?

Shannon:

He was interested. He somehow didn’t always believe these . . . my ideas . . . he didn’t believe they were in the right direction. I got a fair amount of negative feedback almost.

Price:

Well, if you had told him the Second Fundamental Theorem, he might not have believed that . . . nobody would believe that. But maybe you weren’t ready to announce that.

Shannon:

No. Well, I’m not sure I could have stated it very accurately then. We visited England a few years after that.

Mrs. Shannon:

1950.

Shannon:

1950, and there was a convention of engineers, and so on. And Turing was there like many others. [Donald M.] Mackay is another name which is involved in this, I remember.

Price:

Donald Mackay

Shannon:

Don Mackay

Price:

Yes, because it, I recognize that because it was the Macy Conference. And Wiener, and von Neumann there with you at the Macy Conference.

Shannon:

Yes. Well, they were not at this English Conference . . .

Mrs. Shannon:

That was an Information Theory conference.

Shannon:

That was an Information Theory one. But anyway, while there we went over to Turing’s Laboratory in Manchester at the University of Manchester, and he was, his lines crossed mine several times. And he was interested in programming a computer for chess, which I’d had . . . it was a problem that had interested me a great deal. And he was working away with programming a computer at this time. And he had an office up there, and there was a computer downstairs. This was in the early days of computers. And it was hard to tell what a computer was doing here, whether it was misbehaving or what was going on in a computer.

Price:

Right. But that’s a little bit later than the era I’m really interested in.

Shannon:

Well, let me finish the story though, because I want to get to the punch line.

Price:

OK, please.

Shannon:

So I asked him what he was doing. And he said he was trying to find a way to get better feedback from a computer so he would know what was going on inside the computer. And he’d invented this wonderful command. See, in those days they were working with individual commands. And the idea was to discover good commands. And I said, what is the command? And he said, the command is put a pulse to the hooter, put a pulse to the hooter. Now let me translate that. A hooter is an English, in England is a loudspeaker. And by putting a pulse to it, it would just be put a pulse to a hooter. Now what good is this crazy command? Well, the good of this command is that if you’re in a loop you can have this command in that loop and every time it goes around the loop it will put a pulse in and you will hear a frequency equal to how long it takes to go around that loop. And then you can put another one in some bigger loop and so on. And so you’ll hear all of this coming on and you’ll hear this “boo boo boo boo boo boo,”[CS vocalizing in a sing-song fashion] and his concept was that you would soon learn to listen to that and know whether when it got hung up in a loop or something else or what it was doing all this time, which he’d never been able to tell before. That was a great idea, but I don’t think it was really a very good idea. That command seems to have disappeared from the vocabulary. [laughs]

Price:

Well, it certainly shows he’s pragmatic.

Shannon:

Oh, I thought it was a great idea at the time. But I always remember, put a pulse to the hooter.

Price:

Right. But now during the war, I mean you had these wonderful theorems and discoveries and computability and so forth. But you sensed that when he came over here, you talked on this conceptual academic level, you sensed that he was up to very important war business?

Shannon:

I had no concept of the Enigma machine, is that it? I didn’t know of that nor that he was a crucial figure in it. No.

Price:

You knew of other people that you inferred were interested in cryptography, but you didn’t even know he was interested in it.

Shannon:

Oh, no, I knew that he was interested in it, and I thought he was working

Price:

Cryptography?

Shannon:

In cryptography, yes.

Price:

You knew that much?

Shannon:

I thought that, yes. Or I don’t, didn’t, wouldn’t say I knew it, but that was my understanding. But I didn’t realize that he was as important as he was.

Price:

That was a question you didn’t want to ask him, I suppose. Having guessed that he was in cryptography, you wouldn’t want to ask that question?

Shannon:

No.

Price:

Right, because it was too sensitive to ask, perhaps?

Shannon:

Well, in the wartime you didn’t ask too many questions.

Price:

Yes, I understand. Well, he, and this is where Professor Good is very interesting, because he worked closely with Turing. And Turing had brought the entropy measure into cryptography, in England, as early as 1940. And Good published on this in his book on “Weighting of Evidence,” in 1948. But he alluded to it very obliquely. That is, Good did, because it was still under super secrecy. And it wasn’t until 1974 that this could start to be talked about.

But the entropy measure was in there as a measure of fitness and testing on cryptographic cases, and Good knows all about this fortunately. This probably will come out of this M.I.T. meeting, and a book further elaborating on this general cryptography situation in England that’s called the Bletchley Park area, where they did the, it’s just been written by a, it’s just been published by a man up in Newburyport [Massachusetts], who used to work for MITRE for a long time. He was over there with Turing and with Good.

Mrs. Shannon:

Do you know the name?

Price:

This is called The Hut Six Story, by Gordon Welchman, it’s just come out.

Mrs. Shannon:

That’s still another one. I’ve just read two others.

Price:

Please read it because Turing figures prominently in that, and furthermore it contains all the references back to the earlier important ones like Bodyguard of Lies was sort of the 1.

Mrs. Shannon:

and The Man Called Intrepid

Price:

[continuing] And but now, there’s one called The Ultra Secret too. Well, it fits right into your background, but see, I thought I might come over tonight being able to tell you that secrecy was now lifted and you could talk freely about what Turing told you.

And it’s only the past couple of days that I’ve been told that you never had the need to know in the first place, so Turing would never have been able to tell you this. So I had the notion that perhaps since Turing had already come up with the entropy measure in the context of cryptography in 1940, that again that might have been a lead-in to you, because it is known that he got together with you.

Shannon:

Not at all.

Price:

It’s very interesting.

Shannon:

We talked not at all about cryptography. Now I don’t think we exchanged word one about cryptography. We talked much more about things like the human brain and computing machines and that sort of thing.

Price:

Now this is over 40 years ago and I suppose you couldn’t remember if he even dabbled in suggesting that entropy might be an interesting quantity or anything like that?

Shannon:

Not to me.

Price:

He talked all very qualitatively about information but not too quantitatively.

Shannon:

Mmm-hmm.

Price:

OK, well that . . .

Shannon:

That kills that theory.

Price:

Well, I’m glad I had the opportunity to ask, same as about Wiener, you know, because one could easily jump to the conclusion that since you were in American cryptography, he was in British cryptography, that there could have been a mutual recognition of the information measure there.

Cryptography researchers, WWII

Shannon:

Well, that . . . you bring up Tukey here. Now I didn’t know that he had any connection with cryptography at all.

Price:

He only knew that you didn’t . . . I don’t think he said that he had access to cryptography. He just knew that you didn’t have access to it. Whereas Bennett definitely was in the cryptographic world as I guess Nyquist was later too.

Shannon:

Which Bennett is that?

Price:

William R.

Shannon:

Yes, Bill Bennett. I know . . . I knew him.

Price:

He worked closely with Potter and with Mathes.

Shannon:

Yes, right, I knew him. All right.

Price:

So he had full access to cryptography, and he knew that you had no access at all. I think that was the first . . .

Shannon:

I think he’s your most reliable guy on that probably. Or Potter.

Price:

Potter’s dead. But Bennett is going to write something up about this now that he knows that the secrecy is all over.

Shannon:

I wonder if John Riordan is still alive?

Mrs. Shannon:

He was a couple of years ago.

Shannon:

You have . . .

Price:

Now I’ve been told that Riordan was in the cryptography business too.

Shannon:

Yes, I’m not . . . I think he was, but it’s like with — well, all these people, I didn’t know for sure. Some of these things you’d see who they were talking to or whatever, and one thing would lead to another and you’d form your conclusions but this was wartime and you didn’t make . . .

Mrs. Shannon:

Well, you knew where their office was, and there was a whole club, you know. I mean . . .

Price:

Sure. Of course, you aren’t saying that Riordan had any use for entropy are you?

Shannon:

No.

Price:

Not Tukey. Tukey never did anything on entropy. He told me today he never did anything on entropy. He knew about it but he never was interested in it. And Turing, however, was, entropy was pretty much number one with him, you see.

Shannon:

Well, see, that’s a very funny thing, because I never had connected him with entropy at all. You talk about . . . 1940 —

Price:

1940, they were anticipating Hitler. They were starting to work on the code breaking even before ’40.

Shannon:

Well, that’s very interesting but I had never heard anything like that.

Price:

Well, it’s very important to get this down, because I say, you could jump to conclusions so easily. Thanks. Gee, this is really great to get a firm negative, you know. Because there may never been another opportunity, really. Coincidence, people can say, oh this is — you were very close to Turing, obviously, so people could read things into that. I mean Turing’s not alive anymore, so I mean you won’t get another chance, right?

Shannon:

Yeah, that was a funny thing. Well, you’ve probably read —

Price:

How he went?

Shannon:

Well, yes, and his mother’s biography of him . . .

Price:

Making his own cyanide.

Shannon:

Well, yes, and it’s generally believed he committed suicide. Although his mother in her biography of him has said no.

Mrs. Shannon:

Swore not.

Price:

You know, what Good says, he has written this in the History of Computing in the 20th Century, this is a book by Metropolis that’s going over the early work in computers, and brings in the British cryptography work. He says, Good says, it wasn’t until after the war that it was learned that Turing was a homosexual. Good thing it wasn’t until after the War, because if it was known during the war he might not have gotten clearance and England would have lost the war. That’s a very interesting comment. [laughter by all]

Mrs. Shannon:

That whole Bletchley story is fascinating. I read it, and then there was another one . . . oh, I can’t remember the name of the other one, both dealing with the whole situation there. They’re fascinating.

Price:

Yes, and it’s had these interesting mathematical aspects to it. Well, that Turing was able to do this kind of real nitty-gritty work. He was very applied in that area. He and Welchman, Welchman is the man who lives up here in Newburyport, the two of them cracked this Enigma thing wide open with two algorithms that they came up with. And Good says that, well, one of them is due to Welchman, and the other one is due to Turing and only Turing could have discovered it. That’s a little tribute to Turing. I don’t know how it works. But it was only Turing that could have conceived of this. Well, to make a parallel it sounds like the Second Fundamental Theorem or something.

Shannon:

I got involved in cryptography at a later period. I was a consultant. I probably should . . . I don’t know . . .

Price:

You’re talking about NSA now, probably the Advisory Board?

Shannon:

Well, I was invited . . . I think, I don’t know that I have any . . . even though this was a long time ago, I’d better not talk about . . .

Price:

No, but you see, I thought that you might be feeling the same way about Ultra, because I had assumed that you knew about Ultra during the war and that you felt that you couldn’t talk now about Turing. But you know, that’s . . .

Shannon:

Well, since I know nothing about Turing I can talk a lot about him . . . [laughs] That’s the way books get written. Well, I think Turing had a great mind, a very great mind. And his, some of his concepts were terrific, the universal machine. Did you see, I have a little paper which cuts the universal machine down to two states . . .

Price:

All I know is that it looks simple but it’s extremely deep. I mean the concept of the computability with the universal machine.

Shannon:

how . . . [laughs] I think Marvin Minsky realized that was kind of weird. He spent a lot of time trying to find the minimum product of the number of letters you need to take and the number of states inside the machine. He got that down to something like 30 or so.

1949 IRE publication and military applications

Price:

Let’s see, I’m not going from any notes here. I was just hoping to bring these things up. So I’m sure I’m going to forget some important questions. But now this is getting more towards my military interests. And it’s not cryptography. I’ve never worked in cryptography. But I will say this. I thought things had sort of come full circle because I do work in military anti-jamming systems.

And that has some roots in Information Theory and since Information Theory maybe has some roots in cryptography thanks to your work during the war, in a way it’s military back into military through Information Theory. But in your 1949 paper, well, you do this water pouring. Now I guess you don’t recall what I’m talking about there, but you had W*log [1+ (S/N)] for the flat-noise channel, bandwidth-limited of course, and then you had the signal, flat noise too. And that’s in this book. But then you say the colored noise, you imply the colored noise is an unsolved problem. To get the capacity for the colored-noise channel, the non-white channel. But you solved that problem in the 1949 Proceedings paper If you’re given a certain amount of power it’s like water that you will, it’s always positive valued. And you distribute this power around in pockets where the noise is low. Fano maybe called it water pouring. Maybe you’ve never heard that nomenclature . . .

Shannon:

Not until tonight.[laughs]

Price:

I’ve heard it before that people hadn’t heard it. That’s M.I.T. jargon for your algorithm. But you weren’t able to solve it at that time. Apparently you solved this variational problem a little bit later and realized that it was an on-off type of problem. Either you put no power or you put the powerlevel. So it’s like water in pockets. OK. Well, then you ask yourself the question, what is the worst possible interference, that is to get the lowest capacity for a given bandwidth, a given power of the noise and the power of the transmitter.

Shannon:

Interesting question. What did I decide it was?

Price:

You decided it was the flat case.

Shannon:

Sounds right. [laughs]

Price:

And you said something about whereas this would seem to prove that white gaussian noise is the worst kind of gaussian noise, it’s the worst of all noises. And you proved that a little later through entropy power or something like that. Nonetheless, that kind of solution is a Game Theory solution. What’s the worst possible noise and what’s the best strategy, or the worst . . . and I looked at that, you know, 30 years later I wonder about what your motivations were there? Is it conceivable they could have been military? And I asked Brockway McMillan about that and he said, no, you were always just fascinated by minimax problems. [CS laughs heartily] I was reading into that, that that’s an optimum Game Theory, jamming/anti-jamming strategy. But you would go along with McMillan that that was an interesting exercise in minimax?

Shannon:

Well, yes, who can say what one’s motives are?

Price:

But you can say at that point you weren’t interested in military applications, were you, in the ‘40s, the late ‘40s after the war?

Shannon:

No, I have been very seldom interested in real applications. I’m more interested in the elegance of the problem. Or the, so to speak, the kind of, is it a good problem? Is it an interesting problem? I don’t . . . I make my money on the stock market. I don’t make it by proving theorems.

Price:

I know but that’s a form of war, the stock market. [laughter]

Shannon:

Ah, there my thinking changes.

Price:

All right, good. I appreciate that. But what I want to then ask you is, although you were interested in military problems during World War II naturally, and I guess you were pleased to see if they could be applied, it wasn’t a driving force to you to actually make a military gadget particularly?

If your cryptography work wasn’t immediately applied, it didn’t bother you? As long as you got a greater understanding out of it, that was satisfactory to you? And the case could always be made that potentially it was of military importance. But you didn’t see yourself ever in the front-line of military work?

Gun directors

Shannon:

Well, nearest to the front-line would be the anti-aircraft directors.

Price:

Good. Yes, I see, that was certainly very close.

Shannon:

Those things were actually built and I had a hand in those.

Price:

Close to the hardware there.

Shannon:

Yes. And importantly too, because those very anti-aircraft directors shot down the V-1’s and V-2’s . . .

[There were] many different things in World War II, and like Turing’s thing which was very important, breaking codes. But this also was a very important thing. Now I don’t quite take credit for this. I think there was a whole gang of people there, but I was one of them.

Price:

The gun directors.

Shannon:

The gun directors, in particular the anti-aircraft things, which were much more useful in knocking down these V-1’s and V-2’s than for knocking down airplanes. And that was a much more important mission they did. And they were sitting there on the edge of the [English] Channel, and the bombs would come over and only a fraction of them ever got to London or to places like that.

Price:

Well, that SCR-584 radar played a strong role in that, too. They probably were interlocked, the gun director and the radar, yes?

Shannon:

Oh yes. Yes. Well the radar would say where they were, and the…

Price:

Well, that was a great victory for the NDRC, because a couple of divisions cooperating on that, your gunfire control division and the Rad Lab, you know, Ivan Getting’s.

Shannon:

Yes, right.

Jamming/ anti-jamming applications in WWII; 1949 IRE publication

Price:

So that was as close as you got, right, that was as close as you got to the front-line military. And after the war, could I ask, you were never interested in jamming/anti-jamming problems? Can I say that? Can you ever recall being interested in jamming and anti-jamming?

Shannon:

Only at the theoretical level indicated in that paper. Well, that is, not working for any government agency.

Price:

No, no, but now, I mean that’s interesting. I mean that paper that I’m talking about, in the Proceedings of the IRE, January ’49, where you do this minimax solution . . .

Shannon:

von Neumann’s theory of games which came out in ’45 or so.

Price:

Well, you cite it in your cryptography report as one of your references there.

Shannon:

Oh yes. Oh yes. So this whole, any concept like that is, the theory of games was very influential on me.

Price:

Oh fine, the theory of games for cryptography, yes. Sure.

Shannon:

For cryptography and for communication or anything you want to know.

Price:

OK, but now I worry that I may have asked too leading a question because I said, you have this water-pouring minimax problem and now you’re suggesting to me that in fact you did have a jamming and anti-jamming in the back of your mind.

Shannon:

Oh, I think the only meaning of that problem that you mentioned of what is the worst noise and the best answer to that is . . .

Price:

The only meaning?

Shannon:

Well, at any rate, the only sort of application might be . . .

Price:

Application. But had that application in fact, do you think occurred to you at the time or was it just a problem for its own sake in minimax which is what the impression I got from Brock McMillan?

Shannon:

Oh, oh, I don’t know. I would think that I would have thought of that as a case where this, where you’d want to do this.

Price:

You think you would have, but you can’t reconstruct it.

Shannon:

No.

Price:

Well, were you aware during World War II that a lot of jamming and anti-jamming was going on?

Shannon:

I don’t . . . I really don’t know. I can’t recollect whether, whether that had come to my attention or not.

Price:

Well, I know that you didn’t have crypto clearance during World War II. Is it possible that you didn’t even know that radar existed during World War II?

Shannon:

Oh, I knew that, sure.

Price:

It was secret.

Shannon:

Oh, I knew it.

Price:

Did you know that radars were being jammed?

Shannon:

[following a long pause] I don’t recall so. No, I don’t think so. I would say probably not. But I’m not sure. I think that John Pierce and Jerry Wiesner were both deep in radar already in those —

Price:

Yes.

Mrs. Shannon:

John was.

Shannon:

Yes, John Pierce, he was a close friend of mine. And Jerry Wiesner here at M.I.T., the whole . . .

Mrs. Shannon:

Radiation Lab.

Shannon:

Radiation Lab here was very deeply involved in it. And I mean I wasn’t consulting with them or anything but I knew about these things. These things are —

Price:

But jamming and anti-jamming was purposely excluded from the Radiation Lab. That was assigned to Harvard’s Radio Research Lab, up the street.

Shannon:

Oh. Yeah.

Price:

Did you have any contacts with Terman’s organization? That was Division 15?

Shannon:

Fred Terman?

Price:

Yes. He headed the Radio Research Lab which was very much concerned with jamming and anti-jamming of . . .

Shannon:

No contacts.

Price:

No.

Shannon:

Nor did I have contacts with the Rad Lab except by friendship and that I knew the people.

Mrs. Shannon:

In fact, I don’t even remember hearing about the Terman group down at Bell Labs.

Shannon:

Betty [Mrs. Shannon] was working at Bell Labs at that time, and she was working with Pierce’s group.

Mrs. Shannon:

Well, I wasn’t directly with Pierce. I was with Jim Fisk’s group.

Shannon:

Jim Fisk’s group. He was into that, wasn’t he?

Mrs. Shannon:

He ran that whole group . . .

Price:

I know he was high up.

Shannon:

A group of, wasn’t that group in microwave transmission and radar and so on?

Mrs. Shannon:

Oh yeah. Yeah.

Price:

Well, this activity in electronic countermeasures or jamming came along a bit later than the Rad Lab, and instead of, the Rad Lab had so much work to do already in regular radar that they decided to set up a completely different operation at Harvard to take the burden off M.I.T.

Mrs. Shannon:

Yeah, we had a lot of contact with Radiation Labs, but not with [Harvard].

Price:

The Radiation Lab was set up mostly with physicists, whereas Harvard then corralled the remaining electrical engineers to work on the jamming/anti-jamming problem as I understand it. But I’m probably trying to reconstruct too much post-facto here on this minimax problem. It’s possible that, you know, jamming/anti-jamming would make a sensible application, but it probably wasn’t your driving force

Shannon:

Oh no.

Price:

…of presenting that paper at that time.

Problem-solving as research strategy

Shannon:

Bob, let me put it this way, that I think you impute a little more practical purpose in my thinking, than actually exists.

Price:

Well, it’s a trial balloon. I just want to make sure that there wasn’t.

Shannon:

Audio File

MP3 Audio
(423 - shannon - clip 4.mp3)

Well, let me put it this way, that my mind wanders around and I will conceive of different things day and night, as Betty will attest. And the, like the science fiction writer or something like that. I’m thinking what if it were like this, or what is this problem, or is there an interesting problem of this type? And I’m not caring whether somebody’s working on that or whether Washington would care one way or the other, or anything of that sort. It’s usually, I just like to solve a problem. And I work on these all the time. And this problem of the best strategy against the worst noise and so on is just a thing that would occur to me. And in thinking about how would you handle this kind of noise or that kind of noise I would say, well, what would be the worst kind of noise? Suppose some freak were here trying to do this the worst possible way and construct noise, then what do you do and what is the interplay of this. And especially in the background of von Neumann’s theory of games, which had just about then come out, I believe.

Price:

Yes, of course when you drove the point home in your January, ’49 paper, this wasn’t just the worst of gaussian noise, this shape, this is the worst of all possible noise. You were really rubbing it in there. [CS laughs] And finally . . . to tell us gaussian noise is the worst of all possible noises, you know. But even when you’re saying that we’re thinking of that as an academic sort of thing. And of course, being a capacity measure, that is sort of academic theory, using a capacity measure.

Professional collaboration and competition

Price:

But for instance, Brockway McMillan never asked directly for your help on a military problem.

Shannon:

On any military problem? Well, I sure don’t know about that.

Price:

Well, you were real close to each other.

Shannon:

Well, he was right in the next office.

Price:

But he never asked you for help on a military problem even though he was in charge of that kind of thing?

Shannon:

Well, I couldn’t say yes or no. I couldn’t remember. You’re talking a quarter century ago. He may have, if he says I didn’t, I’ll go along with that. Or if he says I did, I’ll go along with that.

Price:

Well, it seems that you worked on problems for their own sake, of understanding and getting the answers without worrying particularly about military applications, or getting sidetracked too much into applications of any kind.

Shannon:

Yes. I don’t want to mix up your flow of thought here, but it’s an odd thing that Brockway McMillan was practically a classmate of mine at M.I.T.

Price:

Yes, he told me that you were latent competitors.

Shannon:

Latent competitors? What’s that mean?

Price:

Well, I guess in the math department. There was a spirit of competition in the M.I.T. Math Department.

Shannon:

Yes, I suppose so. I was a research assistant there, well, or something like that, or a teaching assistant . . .

Price:

That’s about all I can remember that he said. But I think Wiener was carrying on a grand show and there was a certain amount of competition among the grad students.

Shannon:

Well, Wiener was quite a character at M.I.T. We all knew that.

Noise and anti-jamming; noise carrier for CDMA

Price:

Now had it ever occurred to you even though you weren’t you know, involved in applications, had it ever occurred to you that a strategy against a jammer could be to use a noise-like signal, to use that as the carrier, to use noise? You used noise for random coding, of course, to prove your theorems, noise-like signals. But had it ever occurred to you, the idea that we adopted at Lincoln Laboratory to use noise instead of a sine wave, a reproducible noise to combat, to go, more than cryptography now, to combat some sort of interference? See this is what we were tempted to tell you when we were working on it at M.I.T., that were actually going to use noise in an application system.

Shannon:

But you held . . .

Price:

We held back.

Shannon:

What year are you talking about?

Price:

1952.

Shannon:

Good grief.

Price:

Thirty years ago. We thought a) you probably didn’t have the military clearance that we could tell you about this, and b) we weren’t sure you were interested in practical applications anyhow.

Mrs. Shannon:

As a matter of fact, I think he did have the clearance at that point.

Price:

Well, we didn’t know that. But we figured on the other hand you probably weren’t interested in the practical applications. But we’d be the only other people using noise as a signaling method you see, purposely use noise as a signaling method, a cryptographic type of noise.

Shannon:

Yeah, I understand. But I . . . I don’t know about my, I don’t know, or I can’t remember thoughts long [ago] in that specific of detail. Did you talk to John Pierce any?

Price:

Yes, yes, I have. Now that’s the next question.

Shannon:

And also Barney Oliver. They might have some recollections . . .

Price:

I talked to Ed David and Barney Oliver and John Pierce and Dr. Bennett, and Steve Rice and . . .

Shannon:

Oh yes, Steve Rice.

Price:

And Brockway McMillan I’ve talked to mostly. He’s pointed me to other people, George Baker.

Mrs. Shannon:

Bob [Viteau], is he still alive?

Shannon:

He must be gone by now.

Price:

No, I don’t know. Higgins, he’s very old now, but I think he’s still alive, a man named Higgins, he was a vice president, I think . . .

Shannon:

That’s far above my …

Price:

William Higgins. Well, anyhow I’ve talked a fair bit, mostly to Brockway McMillan, some to John Pierce. But it’s on one particular aspect that I did ask them, and I put it in my letter to you and I wonder if you can remember this, though it’s a long time ago. That you did propose using a noise carrier for what we call CDMA, for [Code Division] Multiple Access. You got into this by understanding that multiplexing is the use of orthogonal functions for carriers. And then you went on and said, well, what about, why don’t we use quasi-orthogonal functions and then it suggested itself to you to use noise waveforms.

Shannon:

Brilliant idea. [laughs]

Price:

Reproducible ones. Well, they thought so. They thought so. That is, Brockway McMillan and John Pierce both appreciated what you had suggested.

Shannon:

Yes, I appreciated it too. I thought it was clever.

Price:

Not clever enough to write up apparently. They say it was never written down. Can you remember the event now?

Shannon:

Oh yeah, I remember thinking this and I remember talking to, I think it was Pierce and Barney that I talked to, Barney Oliver. But I . . . but didn’t we have that in that little paper we wrote about —

Price:

Philosophy of PCM, [Proceedings of the IRE, November 1948]?

Shannon:

Yes, wasn’t that in there?

Price:

That’s no little paper, but that, no, it’s not in there. Because this is multiplexing. You’re not talking about multiplexing there.

Shannon:

Well . . .

Price:

It is a very applied idea. It’s very different from Information Theory. You’re actually proposing mechanizing a noise carrier instead of a sine wave.

Shannon:

Yes, it was a great idea. Yes, oh yeah . . . I did this, and I remember this. And . . .

Price:

Oh good. I think it was important enough to remember.

Shannon:

I’m proud of it.

Price:

Even though you never did anything more with the idea.

Shannon:

Look, you should see my attic. I’ve got papers like this that never published.

Mrs. Shannon:

Oh, we’ve got boxes full of unfinished papers.

Price:

Well, I think this one stands out, doesn’t it?

Shannon:

Well, to you —

Mrs. Shannon:

Now that you bring it up.

Shannon:

I could name other papers that have — power . . .

Price:

Well, they thought it was a nice idea. They never . . . the only thing that happened to it was that John Pierce . . .

Shannon:

John Pierce pushed it, pushed that along.

Price:

He pushed it. Two years later he published the idea with Hopper. And then he decided to . .

Shannon:

Yeah, so didn’t Pierce . . .

Price:

that I’ve said, go off in the pulse direction. But he introduces it with the CW continuous noise waveform and then says, oh, this would be too difficult. But that’s exactly what we were working on at Lincoln Lab, reproducing a continuous noise-like signal. Whereas, Pierce and Hopper go off into pulses spaced, you know, and coding spacing like IFF or something. But nonetheless, the introduction there is most interesting, and they, well, both Pierce and McMillan have said that that idea occurred to you about the same time that your Proceedings paper was published. See what they say there —

Shannon:

I think that’s right.

Price:

They say, you had that “some years” ago as of September ’52, and they say it’s early ’49.

Shannon:

Well, I . . .

Price:

That seems to fit, does it?

Shannon:

Well, that would fit with everything I know . . .

Mrs. Shannon:

This was, … it was submitted in November, ’51.

Price:

That “some years” could be . . . OK, it couldn’t be one year.

Shannon:

Although it would fit logically in my thinking, that was about right. That is that’s the kind of idea I had about that time, so to speak.

Price:

Well, that’s as close as you come then to what we were doing at Lincoln Laboratory. It’s using the same system of transmitting on a reproducible noise, but for a different motivation. You didn’t have it as it’s presented there, anyhow, for anti-jamming, where you want to be orthogonal, or semi-orthogonal to nearly everybody else in the channel, because that everybody else might be a jammer, right? You want to be semi-orthogonal to your other friends in the channel, that’s how you presented it. It’s for share . . . sharing the frequency spectrum.

Shannon:

Yeah, even more, a fortiori . . . it seemed like a very democratic way to use up the coordinates you have. [laughs]

Price:

I agree.

Shannon:

I thought . . . and to distribute the cost of living, so to speak, the noise evenly among everybody. It seemed, the whole thing seemed to have a great elegance in my mind, sort of mathematically speaking and even from the point of view of democratic living in this world of communication.

Price:

And furthermore it could be actually applied, unlike channel capacity, it actually could be instrumented.

Shannon:

Yes, even that. [laughs]

Price:

So therefore, a system could, you know, a real world system could . . . have been configured around it if you’d gotten the proper encouragement, more than just everybody saying, yeah, that’s a good idea. But nothing further happened to it.

Now what’s happening these days, the FCC has set up a special docket for this very idea, you see. But in those days, I guess nobody was interested that much in democracy. [laughter] Now the spectrum has gotten more crowded. I get you what you mean about your democracy because it means that a great number of people potentially have access to the same channel.

Shannon:

Oh I love that part of . . .

Price:

As against the greediness of just one person having Channel 5 on television, say . . . right?

Shannon:

Right.

Price:

Yes.

Shannon:

Oh yes, as you bring it back I recall the ideas that more and more people could come and so to speak they would all pay equally if more people are there. But gradually the noise level would increase on each channel. But everybody still could talk. It would be a pretty noisy cocktail party but — [laughs]

Price:

You conceived that just . . ..

Shannon:

Oh yeah, I thought that whole thing was very — I thought that was a very humorous picture.

Price:

That’s very nice. That’s what we call graceful degradation.

Shannon:

Oh, you’ve got a word for it, eh.

Price:

Yeah, that’s military jargon, graceful degradation.

Shannon:

Well, that’s better than some of the military jargon. [laughs]

Price:

OK, well, that was one of my other major questions which had a direct connection with spread spectrum. Actual existing systems.

Shannon:

Yes, I thought, I was more enchanted with the notion that you could continually bring people into this thing, and it would all still be all the same. That everything would be going on the same except that there’s gradually more noise on everybody in the system. I thought that was a wonderful —

Price:

Right. Well, independently this same notion came up at a thing called Project Hartwell at M.I.T. two years after you thought of it, Jerry Wiesner thought of it. But have you ever even heard of Project Hartwell?

Shannon:

I may have, but it doesn’t . . .

Price:

John Pierce’s ideas got into Project Hartwell because Ralph Potter brought them up. He was a participant in this secret Navy military project you know, at M.I.T. And the use of noise carrier was reinvented you might say by two different people, Jerry Wiesner being one, Lou de Rosa of Nutley ITT Labs, being another. To think of the use of noise as a carrier, and reproducing it at the other end. But Jerry Wiesner in addition had this idea of sharing the spectrum, as you say.

And he published it in 500 copies, which you never did. You planted the idea but you never followed through. And so this work, this got spread around the country in the secret circles. And so well, you know how it is with people that publish and people that don’t publish. It makes a difference. But your priority is clearly established, right?, by time, because it’s year ’49, but you never, you kept it private, apparently.

Shannon:

Well, it’s the luck of the game.

Mrs. Shannon:

Well, my guess is that you all sat around one afternoon in the office when you were talking about this and developing it. That you didn’t, you weren’t thinking of it in terms of writing a paper about it. You were writing other things.

Shannon:

It doesn’t really matter. I’ve got enough kudos for my slim appetite anyway.

Price:

Well, I just wanted to tell you that we do consider Jerry Wiesner the prime inventor here, even though you had the idea two years earlier, he came up with it independently and published it in secret circles.

Shannon:

Well, Jerry’s a good friend of mine, and however it works out is all right with me.

Price:

Well, I just wanted to be sure that nothing was written. Because I checked the Bell archives.

Shannon:

That’s —

Price:

The only thing that came up was dated 1952, dated two years later, rather than a year before. So to the best of your knowledge nothing was ever written?

Shannon:

Is this the thing?

Price:

That’s the one, “unpublished work,” it says

Mrs. Shannon:

Yeah, it was submitted in ’51.

Price:

And I’ve had Jim Mazo go and try to find it and he hasn’t turned up anything. And both John Pierce and Brockway McMillan have said, don’t expect to find anything. That you did not document it to their recollection.

Mrs. Shannon:

Well, if there is anything, it’s up in the attic.

Shannon:

Well, something in the attic cannot . . .

Price:

It’s documented very well verbally because there it is very accurately.

Shannon:

But it doesn’t bother me.

Mrs. Shannon:

I’d hate to think of going through those papers in the attic. Oh, you wouldn’t believe, volumes . . .

Price:

Well, the Library of Congress is . . .

Theory of the linear differential equation machine

Shannon:

Well, you know, I have thick papers . . . I have a big theory of the linear differential equation machine, which was published — oh, that’s another thing I did during the war. I’d forgotten when we were talking about that.

Price:

Is that in this book [Gunfire Control], maybe?

Shannon:

No . . .

Price:

There’s a lot about the differential analyzer.

Shannon:

Yeah, but that’s a different thing. I wrote a paper on a differential analyzer and then I wrote a book on linear differential equation machines, which were involved a great deal in fire control equipment. Warren Weaver got me involved in that.

Price:

For the Applied Mathematics Panel?

Shannon:

That’s right. And there was this thick thing, with lots of elegant theorems, I’m very proud of those theorems. But I couldn’t . . . ten years later have published those, because they were all confidential at the time.

Price:

Right. Well, I think that they’re cross-referenced in here.

Shannon:

Show me them. I’d like to see that.

Price:

I’ve seen references to that somewhere.

Shannon:

You’ve got to differentiate these two papers. One was published and the other was restricted.

Price:

Oh, well, these were all confidential, restricted.

Shannon:

Oh, well, then it may well be in there.

Price:

Yes, the NDRC.

Shannon:

Oh, well, then that’s where it might be.

Price:

And I’m pretty sure that . . .

Shannon:

Oh yes, I’d like to see the name of it at least.

Price:

The differential analyzer, that was done at Princeton, wasn’t it? The differential analyzer work, or the difference equation work?

Shannon:

Linear differential equation machine was the name of it, that has to do with that. But . . . it might have been partly in Princeton.

Price:

Look [on page 165 of Gunfire Control, The Theory and Design of Linear Differential Equation Machines by C. E. Shannon at Bell Telephone Laboratories, January 1942]

Shannon:

That’s the thing, yes sir. That’s it.

Price:

And that was done for the Applied Mathematics Panel, but it says Bell Labs.

Shannon:

January ’42.

Price:

It is a Division 7 report.

Shannon:

That’s it. I have a copy of it. Can you get copies of these things?

Price:

These are on microfilms at the National Archives. All these reports are available from the U.S. National Archives.

Shannon:

They’re Unclassified now?

Price:

Oh, everything’s Unclassified.

Shannon:

Interesting. I’d like to someday get a nice little library, essentially about . . . all the things I’ve written. [laughs]

Price:

Well, you’d need to hire a librarian to help you.

Shannon:

I think you might be that guy.

Price:

I have to draw the line somewhere.

Shannon:

I can see. But you’ve done a hell of a lot of work here.

Influence of noise carrier idea

Price:

I should mention also that what Brockway McMillan told me that if you came out with your idea of noise carrier, which is very different from use of noise waveforms for random coding proofs, and Fano has always emphasized the difference of using noise as a carrier in a multiplexing as against gaussian noise signaling waveforms for theorem proving.

Well, anyway, Brockway McMillan told me that that inspired him from his military position to think of the use of that for anti-jamming and also for hiding the signal by spreading it very low and thinly, and reproducing it at the receive end and correlating it and just as you would for your multiplexing because after all, it’s the inner product business. And then, and he thought about that but he decided not to go after that military business at Bell Labs. He saw the potential, but he decided not to push it. That’s why I asked you if he had ever asked you to consult on a military project. Because it could have been that, you see, anti-jamming. And then he told me that a couple of years later he went up to Lincoln Lab and saw what Paul Green was doing there in building such a communication system.

And he got a sort of sense of déjà vu out of it because he’d already conceived it. But just as you did, you both had the idea but you didn’t do anything with it. And so at Lincoln Lab we never got the idea from you or Pierce. It was already underway. See when this was published in ’52 we were already underway. Rather, we got it from deRosa, who brought the idea to Project Hartwell, and he had apparently ginned it up.

But people started thinking that way after your, after A Mathematical Theory of Communication was published in ’48. Then people started thinking about the use of noise-like waveforms. So it came indirectly out of this book, The Mathematical Theory of Communication. But the idea of using a noise carrier was inspired independently of your own idea of using a noise carrier. And maybe your idea of using a noise carrier wasn’t even inspired by this. Do you remember?

Shannon:

Well, by this, meaning what’s in my head, that’s a silly question, Bob. What, how can I say what was, if that was all in my head.

Price:

Well, the way it was presented was that you were thinking about multiplexing in terms of orthogonal signal functions. And the thought occurred to you that noise would be a semi-orthogonal signal function with respect to another noise over a long period of time. That could have come out of . . . that could have come out of the left side of your head while the Mathematical Theory of Communication was coming out of the right side of your head.

Shannon:

Well, uh.

Price:

Because noise as orthogonal signals doesn’t come into this particularly. The noise doesn’t have to be orthogonal.

Shannon:

I see all this stuff sort of geometrically. At any rate that’s my first concept of these things. I see vast, very high dimensional spheres with random vectors around. And — . . . .[End of transcript]