The H-bomb and the computer, part II: the revolution that almost didn’t happen

In spite of the tumultuous development of computers, the architecture in the form of the fetch-execute cycle has remained the same from EDVAC design of 1945 to the present day. And we are used to call this basic architecture “von Neumann machine”. This makes John von Neumann a sort of patron saint of our field.

Not everyone accords this status to the great mathematician who lived from 1903 to 1957. One of my favourite architecture books refuses [5, page 32] to use the term because of the supposedly equal contributions by J. Presper Eckert and John Mauchly. Many of those who do accord full credit to von Neumann for the architecture invoke the name in the pejorative sense of “von Neumann bottleneck” and suggest that the architecture has hindered rather than helped the development of computers.

In this essay I review publications that shed light on the origin of the computer and conclude that it was von Neumann who made the critical contributions in 1944 and 1945. In addition I will argue that von Neumann’s was the basic architecture that propelled the computer along its miraculous trajectory covering three orders of magnitude in size and five in cost and processor speed. Finally I will reflect on the fact that von Neumann was not only an extraordinary genius, but also that he combined in his background a most unusual combination of disciplines — a combination that was essential to the birth of the computer. Without this fortuitous confluence of circumstances the development of computers in the period 1950 – 2000 would not have had the explosive character that we actually experienced. Hence “The Revolution That Almost Didn’t Happen.”

The origin of the von Neumann computer

Although much has been written about the birth of the computer, it seems to me that most of it misses some of the essential points. Of course, like most people alive today, I have to rely on published sources, and easily accessible ones at that. What is remarkable that some of these sources suggest a variant to the mainstream view of events. It is this variant view that I present here.

In the summer of 1944 the physicists in Los Alamos had finished the design of the fission bombs. Especially the implosion device had required extensive and delicately designed caculations. These had been performed by people working with electro-mechanical calculators. A group led by E. Teller had started investigating the possibility of a fusion device (“H-bomb”). Teller’s design was received with skepticism. To better understand the possibilities in this direction a computation was needed on a much larger scale than was feasible in Los Alamos, or indeed anywhere else.

This was the situation on an occasion in the summer of 1944 when von Neumann was waiting on the platform of the railway station in Aberdeen, Maryland. Captain Herman Goldstine, a mathematician working on a secret computer project, spotted the famous mathematician and got up the courage to introduce himself. A relaxed chat followed, as von Neumann was good at making awed youngsters feel at ease. However, when he heard that a computer was being built that was capable of 333 multiplications per second, the conversation became more like a cross examination [1, page 182].

The secret computer project was owned by the US Army’s Ballistics Research Laboratory at the Aberdeen proving ground. The goal of the project was to build ENIAC. There had been electronic computers, but these were analog rather than digital. There had been digital computers, but these were electro-mechanical rather than electronic. ENIAC was to be the computer to combine the speed of electronics with the flexibility and accuracy of digital. By the summer of 1944 design of the ENIAC was complete, and the design group, which included Eckert, Mauchly, and Goldstine, started planning a successor called EDVAC. In the fall of 1944 von Neumann joined the EDVAC discussions on the strength of his lofty security clearance and his position as consultant for both Los Alamos and the Ballistics Research Laboratory.

Eckert, an electrical engineer, found von Neumann a surprisingly congenial discussion partner. Surprising, because they expected a numerical analyst and the topic was logical design. The surprise was a consequence of von Neumann’s unusual background: until 1931 he had worked in David Hilbert’s project in Göttingen, Germany, to provide a logical basis for all of mathematics. Von Neumann moved to mathematical physics when Gödel’s incompleteness result showed that Hilbert’s project, if not doomed, at least needed to be drastically reduced in scope.

Although von Neumann was no longer active in logic, he was sufficiently interested in the work of A.M. Turing to offer him a position extending Turing’s 1936–1938 visit to Princeton. Turing’s paper [6] describing what we now call the Turing Machine had appeared in 1936 and was familiar to von Neumann in 1944. Von Neumann also knew the 1943 paper of McCulloch and Pitts [14] that introduced a notation for abstract switching networks intended to model the interaction of natural neurons [2, page 545, reference to McCullough and Pitts].

McCulloch and Pitts showed that their networks could compute the boolean functions. For good measure the paper remarked that the networks could compute any number that was computable by a Turing machine. All that von Neumann needed to pick up from the EDVAC discussions was that (rather than how) abstract neurons could be implemented in the electronic technology of the time. Von Neumann wrote in McCulloch-Pitts notation a proposal for the logical structure of EDVAC. Much of this was in the form of letters from Los Alamos that were assembled into a report [7] by Goldstine [1, page 196].

By this time a rift had developed in the EDVAC group between Eckert and Mauchly on one side and Goldstine and von Neumann on the other. Eckert and Mauchly felt that they owned the exclusive rights to what was described in the report because the idea of storing data and instructions in the same memory had occurred to them independently. Eckert and Mauchly accepted only such delay in patenting the contents of the report as necessitated by the war-time emergency.

Von Neumann disagreed and regarded the report as the record of a joint effort. He was not in favour of pursuing patents rights for its contents. It seems that his efforts in this direction [8] were as a countermeasure to the earlier proceedings in this regard by Eckert and Mauchly. Von Neumann was familiar with Turing’s 1937 paper which had gone further in the direction of a programmable discrete-state computer than anything the Eckert has claimed to have written. And then there was the 1943 McCulloch-Pitts paper to provide the neuron notation for switching circuits.

Von Neumann saw that the technology of ENIAC proved that the EDVAC design had the potential of providing science with a new instrument of unprecedented power. He hoped for wide dissemination of the EDVAC report. Goldstine proceeded cautiously in this regard, distributing copies to 24 carefully selected recipients [1, page 196]. By the time the patent lawyers for the Ballistic Research Laboratory were on the case, the relatively wide dissemination of the report was cause to withdraw from attempts to patent anything from its contents.

Back to the question: who invented the computer, Eckert and Mauchly or von Neumann? From the above it is clear that the credit must go to whoever connected the inventions of Turing and McCulloch-Pitts to the technology of ENIAC. This qualification is necessary, as ENIAC was, though an electronic digital computer, not a stored-program computer (at least not when the EDVAC report was distributed). In current parlance, it was not a von Neumann machine. Thus the question comes down to whether Eckert and Mauchly should be credited with translating Turing and McCulloch-Pitts into form that could be implemented in ENIAC technology.

Von Neumann seems to have written the EDVAC report as a record of the discussions in which he had participated. He did not say anything about who originated what in the EDVAC discussions. It was Goldstine who edited von Neumann’s notes and directions by telephone into the EDVAC report. It was Goldstine who put von Neumann’s name on the report. Were it not for an intriguing paper published by Eckert [2], it would be vain to try to see behind the veil so discretely drawn by von Neumann over the origins.

Eckert was invited to a conference on the history of computing to make a presentation on the ENIAC, the first digital electronic computer. This gave him no reason to mention von Neumann at all: the design of ENIAC was complete and the building of it well under way before von Neumann even knew of ENIAC’s existence. Yet much of the paper [2] is about von Neumann, reason to read the paper with more than usual attention.

Eckert goes out of his way to cast doubts on von Neumann’s credibility: von Neumann had poor judgment in rejecting minimum latency coding [2, page 533, section 6], he had poor judgment in contending that memories of more than 10,000 words were a waste of money [2, page 535, section 8], he “collected considerable consulting fees from IBM” [2, page 535, section 7]. As far as I can tell, all true. But it does not substantiate the claim that Eckert’s ideas were stolen. Nor does the paper say much what those ideas were.

Eckert’s position is clear: von Neumann took credit for the work of others [2, page 534, last sentence]. From the remainder of the paper it is clear that the main item being disputed by Eckert was the idea of the stored program computer, and that this idea was owned by Eckert because he had thought of it independently. Leaving aside the question of whether this implies that Eckert therefore owned the idea, let us consider the two pieces of evidence presented by Eckert.

As first piece of evidence Eckert presents the sentence in [9] “the operating instructions and the function tables would be stored in the same sort of memory device as that used for numbers”. The date on [9] is September 1945, which is after end of the participation of von Neumann in the EDVAC discussions. Eckert was aware that this was not going to help. To remedy this defect he attached to that report, and also to his paper [2], his January 1944 disclosure of a “Magnetic Calculator”. I cannot find the idea there.

The second piece of evidence is that ENIAC was provided with cable connections for feeding “program pulses” instead of “digit pulses” into the function tables. Eckert states that this was done with the intention to use, at some future time, ENIAC as a read-only stored-program computer:

… there would be no difficulty in doing this when desired. In Aberdeen, Dr. Clippenger later ‘rediscovered’ these uses of function tables, without knowing that they had already been provided for in the original hardware [2, page 529].

Indeed, in 1947 – 1948 ENIAC was converted to such use. It is interesting to compare the above judgment of Eckert that “there would be no difficulty” with the following account [10] of N. Metropolis, who was involved in this work.

Indeed R.F. Clippenger suggested that the function tables be used to store instructions. His idea was to store them as pairs of decimal digits, with each pair corresponding to one of a possible one hundred instructions. The control of ENIAC was to be set up once and for all so as to interpret and execute the instructions. An initial attempt at this scheme by A. Goldstine (wife of the H. Goldstine mentioned earlier) failed because it exceeded the capacity of the function tables. The second approach by Metropolis and K. von Neumann (wife of the J. von Neumann mentioned earlier) succeeded, but only because of a coincidence. On a preliminary visit to the Aberdeen proving ground Metropolis had noticed a many-to-one decoder network nearing completion that was intended for some other purpose. This network made it possible to write an interpreter that did fit into the function table.

There are two possibilities. One is that in 1944 Eckert had seen possibilities that were equivalent to laboriously created software and hardware three years later. The other possibility is that even with the hindsight of decades, Eckert had but a feeble grasp of what it takes to build a stored-program computer.

The first piece of evidence that Eckert presents for deserving the credit for the von Neumann computer betrays a similar lack of appreciation of what is involved. The attributes of the “von Neumann computer” are [12, pages 589 – 590]

  1. single stream of instructions sequenced by an instruction counter
  2. instructions stored with data in an addressable memory
  3. instructions encoded as numbers, so modifiable by arithmetic operations
  4. radix two
  5. word length long enough for scientific computing
  6. single-address, single-operation instructions
  7. single accumulator with a multiplier-quotient register

These features are all present in the EDVAC report. All this was claimed by Eckert and Mauchly to the exclusion of von Neumann. Decades after the report Eckert had the opportunity to reiterate the claim in [2, page 529]. And all we find there are the two pieces of evidence discussed above. And these were only for the idea that “the operating instructions and the function tables would be stored in exactly the same sort of memory device as that used for numbers”.

Eckert’s paper is for me sufficient, though indirect, evidence that the EDVAC report was von Neumann’s work. Burks puts it explicitly and succinctly:

… Pres [Eckert] and John [Mauchly] invented the circulating mercury delay line store, with enough capacity to store program information as well as data. Von Neumann created the first modern order code and worked out the logical design of an electronic computer to execute it. [15, page 312].

The mercury delay line was significant during a period of less than five years; the significance of von Neumann’s contribution, at the logical level, was crucial for several decades.

One should also take into account Goldstine’s opinion, although implicitly manifested: he was there, he participated in every meeting, and it was he who placed von Neumann’s name on the report.

Von Neumann was able to do this work only because the participation of Eckert and Mauchly taught von Neumann how Eckert’s electronic inventions could be used to reduce to practice the logical circuits introduced by McCulloch and Pitts. With the advent of solid-state electronics Eckert’s inventions ceased to be relevant to digital computation. Because von Neumann’s inventions were at the logical rather than the electronic level, they remained the basis of computers in spite of the successive innovations disrupting all other aspects of computing: discrete transistors, integrated circuits, bipolar and CMOS VLSI technologies.

Possibly because of the disagreement that erupted soon after, von Neumann never finished the EDVAC report, alias “First Draft” [7]. Instead he started his own computer project and based it on a further development of where he left the EDVAC design. If it weren’t for Eckert and Mauchly pursuing a patent claim for the EDVAC design as their exclusive intellectual property, von Neumann would have maintained his position that the design arose from joint discussions so that no individual attributions should be attempted. Instead von Neumann had to resort a patent claim on his own behalf [8].

And what are the claims made by von Neumann? One of the first people outside the EDVAC group to be exposed to its ideas was Stanley Frankel, one of those who worked with von Neumann on the first ENIAC computation in 1945. Frankel has stated in 1972 in response to the term “von Neumann machine” that the man himself would never have made that mistake, as he directed Frankel to Turing’s 1936 paper as the first source to be consulted [4, page 10]. The same testimonium [3, page 251] comes from Julian Bigelow, the first engineer to be hired by von Neumann in 1946 for the IAS computer project. As a mute testimony of the importance accorded by von Neumann and his group at the Institute of Advanced Study, George Dyson mentions that the volumes of the Proceedings of the London Mathematical Society from the 1930s are in pristine condition except the one containing Turing’s paper, which was consulted so frequently that the volume became unbound [3, photo section between pages 136 and 137]. A vivid reminder of life before Xerox.

I find it hard to imagine that von Neumann failed to acknowledge the Turing machine in the EDVAC discussions, but Eckert shows no awareness of the relevance of the Turing machine. A crucial link between the Turing machine and EDVAC was the McCullough and Pitts paper [14]. At least Eckert gives evidence that von Neumann must have acknowledged this by mentioning “McCulloch-Pitts neuron notation” [2, section 7, first paragraph].

When von Neumann started his own computer project and continued the design with the personnel of the new project a new document [13] was produced. Blaauw and Brooks have little doubt that von Neumann was the main author [12, page 589]. The contributors are all listed as authors, scrupulously in alphabetic order.

The impact of the von Neumann computer

Witness to the enormous impact of von Neumann’s design is that the “von Neumann architecture” is either taken for granted or is disapproved of in various degrees, ranging from dismissive references to the “von Neumann bottleneck” to the intense loathing evidenced in J.W. Backus’s Turing lecture [16, abstract].

Conventional programming languages are growing ever more enormous, but not stronger. Inherent defects at the most basic level cause them to be both fat and weak: their primitive word-at-a-time style of programming inherited from their common ancestor — the von Neumann computer, their close coupling of semantics to state transitions, their division of programming into a world of expressions and a world of statements, their inability to effectively use powerful combining forms for building new programs from existing ones, and their lack of useful mathematical properties for reasoning about programs.

Backus was not alone in his resentment against the von Neumann architecture. Lovers of APL were offended that their programs ran slower than the Fortran equivalents. The Lisp tribe chafed at the only available architecture. The “Mead/Conway revolution” [17] held out the promise of a shortcut to chip fabrication so that the discontents could actually do something about their grievances. Gerald Sussman used these new opportunities to have a chip fabricated with a processor tailored to the Scheme language [18]. Two companies were launched to produce a computing system based on a hardware architecture customized to executing the code emitted by a Lisp compiler. When a project was launched [whodunnit] for a “Fifth-Generation Computer” with Prolog as programming and database language, a basic premiss was to be liberated from the “von Neumann bottleneck” by an alternative architecture.

While this was happening in the 1980s, Intel chips (implementing, as ever, the von Neumann architecture) and compilers were rapidly becoming more powerful. For the Lisp machines it was the writing on the wall when Goldhill Lisp (of course not comparable in quality to Lisp machine Lisp) ran faster on a vanilla PC than Lisp on an expensive Lisp workstation. The FGCS project was pronounced dead on arrival in 1992 [whodunnit]. The “Mead/Conway revolution” went the way of the backyard blast furnaces during Chairman Mao’s Great Leap Forward.

Blaauw and Brooks explain how every time more and faster transistors have become available, the most effective use has been to render a von Neumann architecture faster. They have a name for the belief that the new hardware technology will do away with the “von Neumann bottleneck”: “next-technology fallacy” [12, page 147. That, in combination with improved compiler techniques, provided the most practical route to the Nirvana that Backus pined for, a Nirvana in the form of languages like Haskell, OCaml, and Prolog. The vast gap between electronics and Nirvana languages needs to be bridged by several levels of abstraction. The von Neumann architecture is useful as insulation from the idiosyncracies of electronics and compilers as insulation from the dread “word-at-a-time style of programming”.

It is remarkable that this model has turned out to be optimal for half a century, from 1945 onwards. Only in the mid-nineties when the number of transistors on a single chip ceased to be the true bottleneck, the “von Neumann bottleneck” may have ceased to be the optimal solution. For the first time after fifty years of progress at break-neck speed, there was a glut of switching elements. This may have made it practical to consider architectures customized to high level languages. But it would only be competitive at the expense of investments that only those firms can afford that are committed to the von Neumann bottleneck. So now, instead of a single such bottleneck, my next computer, with it quadcore chip, will have four of them. And I will be able to have my OCaml programs run wondrously fast. Will all those millions transistors be used efficiently? Of course not: the famed “Road Ahead” is paved with waste.

Coincidences

That brings me to the final topic: that this incredible revolution was such an unlikely outcome. Von Neumann was a genius the like of which his century has seen but few. But the von Neumann architecture of 1945 depended on something else as well: the freak combination of backgrounds that came together in von Neumann. He was an important contributor to logic, the purest of pure mathematics. He got access to ENIAC as an expert in that most applied part of applied mathematics that borders on physics.

Another freak coincidence was ENIAC. When that project was started, the authorities in automatic computation were Aiken at Harvard and Stibitz at Bell Labs. The received opinion was that vacuum tubes were not an option for automatic computers, as nobody had gotten a device to work reliably with more than very few hundred tubes. And yet the Ballistics Research Laboratory was funding a huge project for a computer that was planned to have five thousand. Moreover, the Chief Engineer [19, page 349] was to be a twenty-four year old (Eckert). As soon as the ENIAC design started the requirements escalated: BRL also wanted this and wanted that [2, page 526, last paragraph]. The upshot was that ENIAC ended up with 18,000 tubes. And yet it all worked by the end of 1945, two years after the start of the project. Before the second world war this could not have happened; nor could it afterwards.

To get 18,000 tubes all working long enough to get anything useful done required groundbreaking work, designed on the fly by Eckert. Stern [19] explains the conflict between Eckert and Mauchly on one side and von Neumann on the other as one between cultures: engineering versus science. I think the contrast was stronger; that Eckert was not representative of engineering at that time. By the mid-twentieth century the engineering profession had matured sufficiently to emphasize formal procedures, including the prior production of written designs. Eckert was not only a superb engineer, but he was also representative of the earlier craft stage of engineering. He was an engineer who “thinks with his hands”. In the normal course of events Eckert would have been given gradual increments in repsonsibility and been given the time to mature into a main-line engineer. Only in the war-time emergency could a man in Eckert’s stage of development been given a chance to deploy his great talents in the raw at government expense in a big project. War-time emergency or not, Stern [19] observes that the ENIAC project lacked credibility among the experts in automatic computing. She conjectures that the reason von Neumann was initially so eagerly welcomed to join the EDVAC team was the expectation of increased credibility.

I think that, without von Neumann’s intervention, Eckert and Mauchly could have continued in their intuitive ad-hoc fashion to quickly make EDVAC a success. They would also have entangled the first stored-program computer in a thicket of patents, one for each ad hoc solution. Computing would have taken off slowly while competitors chipped away at the initial monopoly of the Eckert-Mauchly computer company. We would not have experienced the explosive development made possible by the early emergence of a design that, because of its simplicity and abstractness, thrived under upheaval after upheaval in electronics.

The dynamic of the computer’s development has carried us far beyond the question that led to von Neumann’s involvement with EDVAC in 1944-1945. The question was whether Teller’s design for the Super was feasible. Did the new electronic computers help to answer it?

References

[1] The Computer from Pascal to von Neumann, Herman H. Goldstine, Princeton University Press, 1972.

[2] “The ENIAC” by J. Presper Eckert in A History of Computing in the Twentieth Century, N. Metropolis, J. Howlett, and Gian-Carlo Rota (eds.), Academic Press, 1980.

[3] Turing’s Cathedral, George Dyson, Pantheon Books, 2012.

[4] “On Alan Turing and the Origins of Digital Computers”, by B. Randell. In Machine Intelligence 7, B. Meltzer and D. Michie (eds.), Wiley, 1972.

[5] Computer Organization & Design, 2nd edition. John L. Hennessy and David A. Patterson. Morgan Kaufmann, 1998.

[6] “On Computable Numbers, with an Application to the Entscheidungsproblem”. A.M. Turing. Proc. London Math. Soc. Vol. 42 (1936–7), pages 230–65.

[7] “First Draft of a Report on the EDVAC”. J. von Neumann. Unpublished report circulated 1945.

[8] Von Neumann and the Beginnings of Modern Computing. William Aspray, MIT Press, 1990.

[9] “PY summary report No. 1.” J.P. Eckert et al. March 1945. Referenced as note 58 in [8, page 263].

[10] “A Trilogy of Errors in Computing History” N. Metropolis and J. Worlton. Annals of the History of Computing vol. 2 (1980), pages 49-59. I assume that this is the same as a paper with the same title and authors presented at the first U.S. – Japan Computer Conference.

[11] “Amending the ENIAC story” Letter to the editor by John Mauchly. Datamation vol. 25 (1979), no. 11.

[12] Computer Architecture Gerrit A. Blaauw and Frederick P. Brooks, Jr., Addison-Wesley 1997.

[13] “Preliminary discussion of the logical design of an electronic computing instrument” by A. Burks, H.H. Goldstine, and J. von Neumann. Chapter 4, volume V, John von Neumann: Collected Works A.H. Taub, editor, Pages 92-119, Pergamon Press, 1963.

[14] “A Logical Calculus of Ideas Immanent in Nervous Activity” in Bulletin of Mathematical Biophysics, vol. 5 (1943), pages 115–133.

[15] “From ENIAC to the stored-program computer: two revolutions in computers” in A History of Computing in the Twentieth Century, N. Metropolis, J. Howlett, and Gian-Carlo Rota (eds.), Academic Press, 1980.

[16] “Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs” in Communications of the ACM, vol. 21 (1978), pages 613–641.

[17] Introduction to VLSI Design Carver Mead and Lynn Conway, Addison-Wesley 1979.

[18] “The SCHEME-79 Chip” Holloway, Jack; Steel, Guy Lewis, Jr.; Sussman, Gerald Jay; Bell, Alan. AIM-559, MIT Artificial Intelligence Laboratory, 1980.

[19] “John von Neumann’s Influence on Electronic Digital Computing, 1944–1946” in Annals of the History of Computing, vol. 2 (1980), pages 349–362.

Postscript February 8, 2015.

Jimmy Lee of the Chinese University of Hongkong drew my attention to the fascinating account by Amanda Gefter of the collaboration between Warren McCulloch and Walter Pitts (http://tinyurl.com/nf26mms). Their joint paper plays a crucial role in the above story.
~

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: