For brevity you can’t beat the following recap of software history:

Grizzled old hackers tell of going into insurance companies in the 1960s. The typical computer cost at least $500,000 and held data of great value. When Cromwell & Jeeves Insurance needed custom software, they didn’t say, “Maybe we can save a few centimes by hiring a team of guys in India.” They hired the best programmers they could find from MIT and didn’t balk at paying $10,000 for a week of hard work. Back in those days, $10,000 was enough to hire a manager for a whole year, a fact not lost on managers who found it increasingly irksome.

Managers control companies, and hence policies that irk managers tend to be curtailed. Nowadays, companies have large programming staffs earning, in real dollars, one-third of what good programmers earned in the 1960s. When even that seems excessive, work is contracted out to code factories in India. Balance has been restored. Managers are once again earning three to ten times what their technical staff earn. The only problem with this arrangement is …

This is from Philip Greenspun’s “Philip and Alex’s Guide to Web Publishing”, page 318. In case some background is required, read on.

That background is the transition from Craft to Industry. The Grizzled Old Hackers, who at the time were neither old nor grizzled, represented Craft. The managers, in wresting back control, represented Industry. What powered the managers in this struggle is the momentum of several centuries of Industry beating back Craft. The ideology was provided by the Enlightenment philosophers in the 18th century with the movement around d’Alembert’s Encyclopédie, which saw the guilds with their master craftsmen as remnants of the Dark Ages to be conquered by the enlightening forces of Science. Accordingly, the Encyclopédie published manufacturing processes, preferably with scientific explanations, thus far guarded as trade secrets by the guilds.

By and large Industry triumphed. The actors in this process were too busy to notice the large-scale tide of history that they were engaged in. It took outsiders to recognize it and to celebrate it. For example, Richard Buckminster Fuller published in 1962 a book with the title “Untitled Epic Poem On the History of Industrialization”. For another example, let us go to Peter Drucker, who started his career as a management guru in the 1940s with a study of the General Motors corporation.

At the time GM was one of the wonders of the world. Perhaps hard to imagine now, but at the time it was the most successful actor in the most successful paradigm, which was Industry with a capital “I”. On the way to guru status Drucker got a head start out of describing the phenomenon and celebrating its quintessential actor, the Manager. If you ever wondered where the mystique of the MBA comes from, this is it.

A good way to get the flavour of Industry with capital I, and its triumph over Craft, is to look at a specific episode, the Norden bombsight. Up to World War II Germany reigned supreme in optics, an industry concentrated around the firms of Carl Zeiss in Jena and of Ernst Leitz in Wetzlar. Nothing outside of Germany came close. Because of the crucial importance of accurate bombsights, the German military were aware of this and considered optics one of their prime strategic assets. In 1940 they could safely assume that the Allies could not even come close. The German military also believed that this situation was not going to change anytime soon: it would take a generation to produce the highly trained optical craftsmen that produced the high-quality German bombsights.

In the early 1940s the US had a promising prototype bombsight, the Norden. The problem was, in the absence of a suitable optical industry, to manufacture thousands of high-quality copies. Drucker, in his autobiography “Adventures of a Bystander”, recounts how Dreystadt, the head of the Cadillac Division of GM, bid on the nastiest defense contract around, the production of the Norden bombsight. Everybody “knew” that the job required highly-skilled mechanics. At the time there was no unemployed labour, let alone mechanics at any level of skill. Over to Drucker for the delicate wording of this part of the story:

The only labor to be found in Detroit were superannuated Negro prostitutes. To everybody’s horror Nick Dreystadt hired some 2,000 of them. “But hire their madams too,” he said. “They know how to manage the women.” Very few of the women could read and the job required following lengthy instructions. “We don’t have time to teach them to read,” said Nick, “and few would learn anyhow.” So he went to the workbench and himself machined a dozen of the bombsights. When he knew how to do it, he had a movie camera take a film of the process. He mounted film frames separately on a projector and synchronized them with a flow diagram in which a red light went on to show the operator what she had already done, a green light for what she had to do next, and a yellow light to show what to make sure of before taking the next step. By now this is standard procedure for a great many assembly processes; it was Dreystadt who invented it. Within a few weeks these unskilled illiterates were turning out better work and in larger quantity than highly skilled machinists had done before.

This is one of the many defeats suffered by Craft in the past centuries. It was Drucker who developed the ideology of Industry and the central role of the manager in it. Together with the work W. Edwards Deming in quality control, this represented a neat package ready for export. In due time the US market was flooded by innovative consumer items powered by precision miniature electric motors designed by Japanese engineers and assembled by unskilled illiterate Chinese peasant women. The engineers not only designed the artifact, but also the process of manufacturing it, down to the assembly steps and the training of the workers. This is called industrial engineering, a branch of engineering dating back to Frederick Taylor in the early 1900s.

Drucker’s story does not have a happy ending. Dreystadt’s victory over the unions was only a temporary one. When the war was over, the women were sent back to their former lives.

It was against the background of the triumph of Industrial Engineering that the “Software Crisis”, so named by professor Bauer from Munich, was discovered. Not long after E.W. Dijkstra, in his Turing Award lecture, summarized the situation as

… as long as there were no machines, programming was no problem; when we had a few weak computers, programming became a mild problem, and now that we have many powerful computers, programming has become a huge problem.

In response to the Software Crisis the NATO Science Committee sponsored a meeting in Garmisch, Germany in 1968; frugally in October, before the skiing and after the summer tourist season. The title of the report is

SOFTWARE ENGINEERING

At the time the effect was electric: outside the in-crowd at Garmisch the term “software” had barely ceased to be a startling novelty; to combine it with “engineering” was a bit of a taunt, like “artificial intelligence” a decade earlier. The discussion at the conference reveals glimpses of the prevailing mindset. Here is a remark from Douglas McIlroy, soon to become the godfather of Unix:

We undoubtedly produce software by backward techniques. We get the short end of the stick in confrontations with hardware people because they are the industrialists and we are the crofters. Software production today appears in the scale of industrialization somewhere below the more backward construction industries.

Whether the good people at Garmisch were conscious of it or not, they were under the spell of the overbearing success of the Industry paradigm. They took for granted that the way ahead was to follow it. Thus the report has as Chapter 4 “Design”, followed by Chapter 5 “Production”. Note that “Production” means the programming of the system.

The analogy is not valid: when you go into “Production” with bombsights, you have a prototype. A working, tested, documented, prototype. Apparently the Garmisch people forgot that when you have a working, tested, documented prototype of your software system there is only something left to do for the marketing people. Industrial Engineering is a legitimate branch of engineering because of the difficulties that attend the accurate and economical duplication of a working, tested, and documented prototype. To substitute the “Industrial” by “Software” in Industrial Engineering is to fall into the cargo cult trap.

For those who did not hear or read the late Richard Feynman’s CalTech 1974 commencement address, or the quote of it in my essay of August 2008: Feynman explains why, even though they perfectly emulate science, certain studies in psychology are not. All the right moves are there: Null Hypotheses, Type I and Type II errors, …, the works. But though the form of these studies is perfect, the exercise is vacuous. He called this Cargo Cult Science, after the phenomenon observed shortly after the second world war on some of the islands in the Pacific. From Feynman’s address:

During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antenna’s — he’s the controller — and they wait for the planes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No planes land.

Early technology was about providing power (as in horsepower) by machines. When this was well established, the stage was set for Industry, which was about the organization of routine work: assembly lines and all that. This is where Industrial Engineering lives and where Industry has triumphed over Craft.

By the time Industrial Engineering was discovered, clerical work such as the processing of mail orders and bank transactions was done on a large scale. This motivated the organization of clerical work along the same principles as those of mass production in manufacturing. During the second world war scientific computations reached such a scale that the same organizational techniques were needed that had earlier been applied to the processing of mail orders and bank transactions. Enter the computer, and all the carefully engineered routine work was swept away.

The computer introduces a new game. While the processing of material leaves an irreducible residue of work for humans, in the processing of information any work that is routine instantly vanishes. Extracting the routine part from an information processing task is a creative endeavour. It is called programming. In the building of a software system any time you think you have something routine to be handed over to managed cubicle-dwelling drones, you are missing an opportunity for automation. In the building of a software system there is only room for creative work. It is Craft, irreducibly so.

One of the few true bestsellers in the computer world is Design Patterns by Gamma, Helms, Johnson and Vlissides (“The Gang of Four”). Not only did it sell well, but its success also triggered the appearance of dozens of books with title and content conspicuously linked to the original. As the Gang acknowledged, the original itself is, if not an imitation, at least strongly inspired by another book. Their product is highly original in having its source of inspiration in an area far away from computers and software. That source is Design Patterns by Christopher Alexander, Sara Ishikawa, and Murray Silverstein, and the area is design in architecture. Not computer architecture, not software architecture, but architecture. To avoid confusion, I will refer to the Gang-of-Four book as “DP” and accord to the original original the honour of Design Patterns in full.

What is not widely acknowledged, and does not even seem to be known among the aficionados of DP, is that Design Patterns is but a companion to the main volume, which is The Timeless Way of Building with Christopher Alexander as sole author. That the epiphenomenon makes a great furore in the software world while the main item remains unknown is, it seems to me, remarkable. The present essay may be useful in introducing the unknown main item and arguing its relevance for the building of software.

The Timeless Way of Building is a reaction to what may be called The Agony of Architecture: architecture as a profession has achieved unprecedented size and prestige without relief from the awfulness of most new building. In the rare case where a real architect is retained for the design of a new housing development, the designer himself is nowhere to be found there, not even in a penthouse suite designed for himself by himself. Instead he is more likely to live in Greenwich Village, or Siena, or in the vicinity of the Place des Vosges. Bernard Rudofsky rubbed salt into the wound of the Agony by wordlessly documenting the missing ingredient in a book with the tactless title Architecture Without Architects. As samples he presents Phera in Santorini, the hill town of Anticoli Corrado near Rome, Alberobello in Apulia, Dogon (South of Timbouctou), among many other places.

The first, as far as I know, 20th-century architect who did something about it other than voting with his feet, is Christopher Alexander, a professor of architecture in Berkeley. He tried to put his finger on what makes Architecture Without Architects so beautiful and the products of famous architects, though Award-Winning and Magazine-Featured, so disappointing when in actual use after the initial brouhaha has subsided. The result is The Timeless Way of Building.

I have been somewhat unfair to the architects of the 20th century. I am enthralled and uplifted by Saarinen’s IBM T.J. Watson Research Laboratory in Yorktown Heights, by Gehry’s Experience Music Project in Seattle, by Pei’s stairwells in the Musée du Louvre, and Houben and Richter’s Central Library of the Technical University in Delft; with apologies for the eclectic sample — restricted as it is to one person’s experience.

The point here is that there are two kinds of architecture: the creation by the individual artist that draws attention to itself as such, and the lived-in fabric of the wistfully admired old towns of Siena, Lucca, Pisa, the canal district of Amsterdam, among still, fortunately many, other examples. The latter may be called “egoless” architecture, to borrow from “egoless programming”, introduced in 1971 by Weinberg in “The Psychology of Computer Programming”.

What is, then, the genius of Architecture Without Architects? It is, what The Timeless Way of Building calls “The Quality”, which is something that is difficult to describe. To make progress in this direction, we must recognize that the saying

Everything that can be said at all, can be said clearly.
And of what cannot be said clearly we must not speak at all.

is a council of despair. It so happens that it is precisely the most important things about which we are not able to speak clearly. But it helps to try to say something.

Not only has it not been said clearly what The Quality is, but those who have tried are not even explicit. Alexander, in The Timeless Way of Building, proceeds by oblique references, by implications. And lots of photographs, without captions; Alexander is pointing rather than defining. When Pirsig started to teach about The Quality, he ended up with a whole novel, Zen and the Art of Motorcycle Maintenance. Which brings us to Eric Raymond, who tries to explain in The Art of Unix Programming what makes the way of Unix unique. He becomes explicit by invoking the quality of “transparency” in code (page 149) but retreats immediately into the enigmatic:

One of the main lessons of Zen is that we ordinarily see the world through a haze of preconceptions and fixed ideas that proceed from our desireds. To achieve enlightment, we must follow the Zen teaching not merely to let go of desire and attachment, but to experience reality exactly as it is — without the preconceptions and the fixed ideas getting in the way.

With these warnings in place, I reproduce some of Alexander’s pronouncements in The Timeless Way of Building.

THE TIMELESS WAY

A building or town will only be alive to the extent that it is governed by the timeless way.

…

THE QUALITY

To seek the timeless way we must first know the quality without a name.

…

Notice that we are expected to know something without having a name for it.

THE GATE

To reach the quality without a name we must then build a living pattern language as a gate.

…

Alexander’s emphasis on language may need clarification because he skips a step. The town represents a community, a tradition, a culture. The three concepts overlap and interact by being based on a language. Alexander skipped the vague concepts and went straight on to something concrete — a language. The language is not a written one: its words are the visual and spatial patterns, 253 of them, photographed, sketched, and described in Design Patterns. In the abstract world of software there is nothing to photograph, but notice the diagrams in DP: they play an important role.

Recently Michael Levy, a software engineer at Apple, Inc, gave a lecture to a class of undergraduate students in computer science. These students have been trained to see programming as an individual activity: they are expected to do their assignments and write their exams on their own. Levy’s purpose in his talk was to correct this impression. The gist of the talk was that great software is not produced by Great Programmers but by a great community of programmers.

The community stays alive by new programmers joining. They do not replace the ones who have left, no more than babies replace the fallen warriors. Instead, the new arrivals are gradually acculturated. The culture they absorb is partly explicit: formatting rules for source code, naming conventions, policy for allocating and de-allocating dynamic storage. But the explicit represents only the tip of the iceberg; much is absorbed by osmosis. Levy’s message may be summarized as “Great software is not created by Great Programmers, but by egoless programming in a community defined by a culture.”

In 1997, in The Cathedral and the Bazaar Raymond also analyzed software as a community phenomenon. Here is the opening paragraph:

Linux is subversive. Who would have thought even five years ago (1991) that a world-class operating system could coalesce as if by magic out of part-time hacking by several thousand developers scattered all over the planet, connected only by the tenuous strands of the Internet?

The success of early Unix was already an example of the power of community: because of its lack of official status, programmers within Bell Labs supplied as volunteers numerous tools and library functions. The Unix world spread beyond Bell labs in the 1980s, but the power of community did not grow commensurately because of the legal constraints of various licenses, trade secrets, and (imagined) commercial interests.

When the Internet passed the relevant threshold in the early 1990s, Linus Torvalds blew away what had been holding back Unix in the previous decade. Although Torvalds is among the select few individuals who wrote an operating system kernel by themselves, his main contribution was the creation of an Internet-based community.

Linux is a community, self-consciously so since The Cathedral and the Bazaar. The fact that this community has built something significant draws one towards a comparison with the subject matter of The Timeless Way of Building. The validity of such a comparison depends on whether there is in the Linux project a counterpart of what Alexander calls The Quality. Indeed, in 2003 Raymond published The Art of Unix Programming, which can be read as an attempt to communicate by numerous examples what Alexander describes as “the quality that has no name”. Raymond may have wished he could give his book as title “Zen and the Art of Unix Programming”.

There are two kinds of architecture: the conspicuous spectaculars of Gehry et al. versus the lived-in fabric of some of the towns of old. Let us look at one such old town: Siena. As you view it from a distance it shows representatives of both: the cathedral and the town. The difference between the cathedral and the contemporary spectaculars is that the cathedral is not the product a single ego, but was, like the surrounding town, the product of a community — although one differently structured and operating differently from the one that grew the town.

Raymond, in The Cathedral and the Bazaar, writes that a community based development process can add to an existing design and can debug, but that the design, which can only be conceived by an individual, needs to be there at the beginning. It may be that the true test of The Gang of Four will be to transcend this limitation. The patterns in The Gang of Four may turn out to be the nucleus of a store of patterns that provides the self-selected community around an open-source project with the tools of thought and communication that allows such groups to evolve designs that recall, in this abstract domain, the magic and majesty of the cathedral of Siena.

In 1968 the Communications of the ACM published a Letter to the Editor from E.W. Dijkstra (volume 11 (1968), pp 147–148). The Editor, perhaps a bit puzzled by the contents, summarized them by supplying as title “Goto Statement Considered Harmful”. The mere thought was sufficiently provocative to ensure that the idea spread like wildfire, initially to be dismissed, but soon to be blessed with universal approval. “Structured Programming” was born.

The most enthusiastic converts were the computer science professors. These people were doing research on esoteric topics like formal languages or graph theory and were gang-pressed into teaching introductory programming. With little programming experience, they welcomed a clear rule (no GOTOs!) and welcomed the casuistry required for forcing non-conforming code into the straight jacket of the officially approved formats.

This essay is a thought experiment: suppose one had never heard of Structured Programming, that one wanted to solve a little programming problem by common sense reasoning; that this solution process not only had to supply the conviction of correctness, but also had to help discover the algorithm. My conclusion will be that these goals, though ambitious, are achievable, but that Structured Programming is at best a handicap.

Do I then conclude that Dijkstra was wrong in his Letter to the Editor? No, but I do conclude that one should not read as much into it as those in the Structured Programming mania have done. Dijkstra said that there is a positive correlation between the density of GOTOs and the density of bugs. I am prepared to believe that. He could have phrased it as “Lots of GOTOs — Lots of bugs.” He would then have echoed the rule “Bad spelling — bad thinking”, a rule that encapsulates the correlation observed by professors who have had to grade large numbers of student essays. The rule sounds provocative because the spelling and the thinking are at different levels. Surely, one can be a bad speller and a good thinker. Yes, that’s in theory. What the profs found in practice is that this theoretical possibility is rarely realized. What I hope this essay will convince you of is that Dijkstra’s Letter to the Editor has the same kind of validity as the rule “Bad spelling — bad thinking”. That, indeed, the more GOTOs in the source code (as typically found in 1968), the more bugs. This leaves open the possibility that a certain kind of thinking exists that leads to correct code in a process that ignores the precepts of Structured Programming.

The example

First some decisions: (1) What programming problem to solve for a demo? and (2) What language to use?

1. For the problem I picked the partitioning stage in Hoare’s Quicksort. In practice I would not recommend Hoare’s partitioning algorithm: I would prefer Lomuto’s variant for the partitioning method as being easy to get right by conventional methods. But for the present purpose I will profess ignorance of Lomuto’s beautiful invention so that we will need to solve a problem that has been acknowledged as hard by Jon Bentley (“Programming Pearls”, Addison-Wesley, 1986, footnote on page 110):
   

   Most presentations of Quicksort use a partitioning scheme based on two approaching indices, … . Although the basic idea of that scheme is simple, I have always found the details tricky — I once spent the better part of two days chasing down a bug hiding in a short partitioning loop. A reader of a preliminary draft complained that the standard two-index method is in fact simpler than Lomuto’s, and sketched some code to make his point; I stopped looking after I found two bugs.

   It is unusual for a programmer to make such an admission, but then Bentley is an unusually good programmer. Another circumstance pointing to the selection of this task is that it has been selected by Hoare himself for his demonstration of formal verification (“Proof of a Program: FIND” by C.A.R. Hoare, Communications of the ACM, Vol. 14 (1971), pp. 39–45)

   We are not about to code a pre-determined algorithm: more ambitiously, we start from a requirement. We end up with code that demonstrably satisfies it. Presumably there is an algorithm encoded in it, but we don’t need to know it as such. (Dijkstra does something similar with the examples in his book “A Discipline of Programming”.) An advantage of what follows here is that our code is more readily executable than that of Dijkstra’s book. Further description of the problem to be solved has to wait for a discussion of the next point.

2. What language to use? Language is needed to describe the requirements of the function to be programmed and to carry out the reasoning that gives us our conviction of the program’s correctness. Both are informal. There is no dearth of formal approaches proposed in the course of decades of research. None of these has found any response among programmers. In addition to this difficulty there is the fact that software for formal verification is often more complex than the application to be verified and that the formal specifications are as hard to understand and get right as any code.

Formal specifications originate in informal notions. The goal of verification is conviction of correctness on the part of a responsible professional. This conviction is also informal. Thus we are going from informal to informal so that the use of formal verification is a detour. As it is not clear how such a detour would help, we aim at informal development leading to executable code and to conviction of its correctness with respect to an informal requirement. This informality does not preclude the use of algebraic notation referring to a mathematical model.

The computational model

The computational model is based on (computation) states; any state being determined by the values of the variables. (Computational) actions typically change a state. Actions are specified as binary relations, that is, as sets of pairs of states. A pair <s,t> is in the relation iff t is a possible state resulting from the action starting in state s. Binary relations include the identity relation consisting of the pairs <s,s> for all states s. We identify “action” with “binary relation over states” so that, paradoxically, there is an action that is the identity relation, that is, an action that does nothing. In addition, the actions include all subsets of the identity relation.

The composition of two actions X and Y is the action obtained by
executing Y in the state resulting from executing X. We write the
composition of X and Y as X;Y. Regarded as a binary relation, it is the set of all pairs <s,u> such that there is a state t with <s,t> in X and <t,u> in Y.

A set of states can be regarded as a condition by identifying the set with the condition that determines whether a state belongs to the set. Conditions define certain subsets of the identity relation: condition C defines the set of all <s,s> such that s belongs to C. For example, if x and y are variables, then x=y is a condition. If from the context it appears that x=y is a binary relation, then, for example x>y ; (x := y) is a binary relation, hence an action. It is an action that is only possible in a state that satisfies the condition x>y.

We will say “C-state” for a state satisfying condition C.

Conditions and actions are described by assertions and statements, respectively. We are interested in partially characterizing a statement by requiring that, if a state s satisfies assertion P, and statement S is possible and is executed starting from s, then there is a resulting state that satisfies assertion Q. Let us call this relationship between P, S, and Q a triple. Hoare used the notation {P}S{Q}. We may regard S as a non-deterministic operator on states. From this point view there is a similarity between triples and the quantum-mechanical objects denoted by Dirac’s notation <Q|S|P>, where the Q part is called “bra” and the P part is called “ket”. As there will be no confusion with quantum mechanics, we will write {P}S{Q} as <P|S|Q>. That is, we adopt the bars and angled brackets from Dirac, but we stick to the order in which the input is on the left and the output on the right. Although we are just using a superficial similarity between the quantum mechanical notion and Hoare’s triple, we don’t want to rule out the possibility of something deeper, like a common generalization.

Analysis

Our goal is to write a function body that executes the partition phase of Quicksort. The function body is a statement, probably compound, that we call S. It needs to satisfy <P|S|Q> where condition P is:

   indexes p and q bound a segment of a numerical array a
   and p < q

We denote the sequence a[p],…,a[q] as a[p..q].

We write e.g. a[p..i-1] <= m for

for all x, p <= x < i implies a[x] <= m

There is available a function with heading

    void swap(int a[], int x, int y)

that interchanges the values of a[x] and a[y].

The purpose of S is to execute calls to swap() and to find an index j such that the state satisfies the condition Q, which is:

p <= j <= q and
a[p..j-1] <= a[j] and
a[j+1..q] >= a[j]

Moreover, in the final state a[p..q] should be a permutation of a[p..q] in the initial state. As the only changes to the content of a[p..q] will be calls to swap(), this is automatically satisfied. Accordingly, there will be no need to include the permutation requirement in any of the assertions.

It would be nice if we would see right away a simple statement S such that <P|S|Q>. As this is not the case, we look for an intermediate condition I such that <P|S0|I> and <I|S1|Q> with simple S0 and S1. This comes to to finding I as a common generalization of P and Q. One such I is:

a[p] = m and
p < i and i <= k+1 and k < q and
a[p..i-1] <= m and
a[k+1..q] >= m

It may help to draw diagrams for Q and for I.

An S0 satisfying <P|S0|I> is

if (a[p] > a[q]) swap(a,p,q);
m = a[p]; i = p+1; k = q;

We don’t see a simple S1 to satisfy <I|S1|Q>, so we postulate an intermediate I0 such that <I0|S10|Q>. If we define I0 as (I & i >= k) then S10 can be

if (a[i] > m) j = i-1;
// a[j] <= m & a[j+1..q] >= m
swap(a,p,j); return j;

In (I & i >= k) we have k <= i <= k+1, so that i is k or k+1. That is, a[p..i-1] and a[k+1..q] touch or have a single element in between. Again, a diagram may help.

We now have the triples <P|S0|I> and <I0|S10|Q>. If we would have (I => I0), then we could conclude <P|S0;S1|Q> and partitioning would be solved by the program S0;S1. That not being the case, we look for a statement X such that <I|X|I0>. If we define I1 as I & i<k then we can find an X such that <I|X|(I0 or I1)>. As execution of X may result in an I1-state and I1 does not imply Q, we need a triple with I1 as pre-condition and Q, I1, or I0 as post-condition. A promising triple is <I1|Y|I>, with Y equal to

if (a[i] <= m) i++;
else if (m <= a[k]) k--;
else {swap(a,i,k); i++; k--;}

Our triples are now

<P|S0|I> <I|X|(I0 or I1)> <I0|S10|Q> <I1|Y|I>

In a sense these triples solve the partitioning problem. S0 terminates from any P-state and similarly for the triples containing S10 and Y. We haven’t seen X yet, but it will terminate for any I-state. All triples together determine that, for any P-state, one of the sequences

S0  X  (Y X)* S10

performs partitioning. The number of occurrences of Y is finite because Y increases i or decreases k and neither can do so indefinitely.

Synthesis

Though the triples are not executable, they are easily translated to executable code. A triple <A|S|B> translates to

A: S; goto B;

in the C programming language.

We haven’t written yet written the X in <I|X|(I0 or I1)>. In general, a triple <A|S|AB0 or AB1>, where AB0 is (A and not B) and AB1 is (A and B), is satisfied when S is

if (B) goto AB1; else goto AB0;

The translations of the triples should be preceded by

goto P;

and followed by

Q: return ...;

In this way we get as a translation of the triples for partitioning:

goto P;
/////triple://///
I: if (i >= k) goto I0; else goto I1;
/////triple://///
I0: j = (a[i] > m) ? i-1 : i;
    swap(a,p,j); goto Q;
/////triple://///
I1: if (a[i] <= m) i++;
    else if (m <= a[k]) k--;
    else {swap(a,i,k); i++; k--;}
    goto I;
/////triple://///
P: if (a[p] > a[q]) swap(a,p,q);
   m = a[p]; i = p+1; k = q;
   goto I;
/////////////////
Q: return j;

The validity of the triples ensures that when this is executed starting from a P-state, the resulting state (if any) is a Q-state. There will be a resulting state is guaranteed, among other things, by the fact that Y is executed a finite number of times.

Just as there is no order among the triples, the order among their translations is immaterial. But certain orders allow more optimization. The above can be re-ordered and optimized to:

   if (a[p] > a[q]) swap(a,p,q);
   m = a[p]; i = p+1; k = q;
I: if (i >= k) goto I0;
    if (a[i] <= m) i++;
    else if (m <= a[k]) k--;
    else {swap(a,i,k); i++; k--;}
    goto I;
I0: j = (a[i] > m) ? i-1 : i;
    swap(a,p,j);
    return j;

Comparison with Structured Programming

This latter program is not suitable for human consumption. This short partitioning loop could easily hide a bug like the one that gave Bentley so much trouble. If only P and Q were given and if this program were presented as an S to satisfy <P|S|Q>, then the assertions would have to be discovered — the notoriously difficult program verification problem. But in the process described here the assertions were there first, suggested by the specification, jointly with the triples. The programmer can think in terms of assertions and triples, not branches and loops. Compared with this process, let’s call it Assertion-Driven Programming, Structured Programming is a distraction and a handicap, an artificial restriction to the use of canned branches and loops.

Assertion-Driven Programming is not the coding of an algorithm. Yet, in this case, the end product behaves like Hoare’s partitioning algorithm rather than Lomuto’s. Somewhere along the line a decision must have been taken that caused Hoare’s algorithm to come out. This choice is a consequence of the choice of I as an assertion intermediate between P and Q. The intermediate chosen here is not the only one possible; there some other one that would have led to Lomuto’s algorithm.

Let’s represent the unordered collection of triples by a labeled graph. This has the advantage that we can relate Assertion-Driven Programming to Structured Programming. In this graph representation the assertions are the nodes; the triples are the directed edges, with the actions labeling these edges. The computations are paths from P to Q. Of course these paths can be described by a regular expression. The operators of regular expressions are similar to the Structured Programming constructs: star is like while; plus is like if-then-else.

Which is better: the regular expression or the graph? In the graph the vertices are explicit; they correspond to the assertions. The regular expression hides this information. To end up with verified code, we need to start with the assertions, which makes the graph essential. This explains the superiority of Assertion-Driven Programming over Structured Programming.

The assertions are informal. The computational model is mathematically sketchy. These shortcomings are an essential advantage: Assertion-Driven Programming is intended to help a practicing programmer to write correct code. Far from being contradictory, testing of the final product is essential: nothing real is error-free right away. But debugging in Assertion-Driven Programming is different. It does not address the executable code, but it requires the programmer to re-start the entire Assertion-Driven process: to check the assertions, the validity of the triples, and the transcription to executable.

Summary

In Assertion-Driven Programming one starts with a specification in terms of a precondition P and a postcondition Q. These provide the initial assertions. Usually it is not immediately clear which sequence S of statements satisfies the triple.

Intermediate assertions are suggested by the existing ones to make it more likely that the required bridging statements can be found. As soon as the assertions and the triples guarantee that there is a computation from any P-state to a Q-state, the set of triples is complete. The triples are then transcribed to executable code.

PS

There is more in Dijkstra’s letter than the observation that the densities of GOTOs and of bugs are positively correlated. Of course, neither Dijkstra nor his any of his readers will have made the mistake of concluding a causal relationship one way or the other. In his letter, Dijkstra also observed that the human mind is bad at visualizing processes evolving in time, so that it is safer to think in terms of static relationships. Structured Programming still refers to a dynamic model; its advantage is that the dynamic model is simpler. Assertion-Driven Programming takes a more drastic step towards a static model.

Maybe not. It used to be taken for granted that in the more or less remote future it would no longer be necessary to debug software because it had been verified.

That time is long past. What we now take for granted is that, for example, the two pieces of software essential to the health and safety of the internet, MS Windows and MS Internet Explorer, “vulnerabilities” are discovered at such a rate that the repair sessions are saved up to the second Tuesday of each month. Rarely does such a Tuesday pass without an update; sometimes five or six. Another exhibit for the prosecution: many pieces of software that invite being downloaded distinguish between the “latest version” and the “latest stable version”. Forget correctness: “stability” is the goal. Diminished expectations …

For smaller programs, such as device drivers, one can still find people aiming at verification. Two approaches have been considered: formal and informal. According to formal verification the program and its desired properties are stated as theorems in a suitable formal system. All that is needed then is to prove the theorem. The attraction of converting the program and the properties to a formal system is that such a proof can then be carried by a program. Let’s call this program a mechanical prover, to cover both inference systems and model checkers.

An example of the kind of difficulty encountered here is described by Martin Ouimet in his paper “Formal Software Verification: Model Checking and Theorem Proving” (MIT Embedded Systems Laboratory, 2007):

“A hurdle to usability is the size of the proof system and the size of the proofs that are being derived using mechanical provers. As a sample case study, a verification approach conducted at Motorola utilized a proof system with relevant concepts spanning hundreds of pages. Furthermore, on the way to the proof of a theorem, it is possible that hundreds and potentially thousands of lemmas must be proved along the way. In the Motorola case study, the size of a proof of a meaningful theorem is stated to require 25 megabytes of memory to print.”

Note that a good-sized novel takes up about half a megabyte. So the situation seems to be that you feed the mechanical prover the inputs, you go out for lunch, and find on return an enormous amount of unintelligible output. This purports to be a proof. You skip to the end, where it says: QED.

The question arises: does this episode raise your confidence in the program subjected to this treatment; does it help you to make the decision whether it is justified to include it in a safety-critical application such as a controller of an aircraft or of a cardiac pacemaker?

For starters, the verification program itself is not verified. Before we even get to the (unintelligible) proof, the concepts used in expressing the desired properties have to be formalized. In the report mentioned, Ouimet speaks of hundreds of pages of Lisp to express the concepts. Verifying these may not be easier than verifying the software itself.

Compare formal verification, as just sketched, to some informal verification methods that have been tried long ago. What I have in mind are the Code Inspections in the so-called Cleanroom Method of software development. According to this method, programmers are not permitted to run their code. Instead, they are required to revise it until they are confident of its correctness. This confidence can only be based on understanding the source code. This understanding is then tested in an inspection, a formal meeting with a few peers, in which one of these goes through the code paraphrasing small bits at a time leading an attempt by all present to detect errors. This practice has been shown to be effective in improving the quality of software compared to the still prevalent approach where no inspections are conducted, and where confidence is gained by failure to find faults during testing.

This is the contrast I want to consider in this essay: on the one hand the approach via formal verification, which turned out to be costly and fruitless. On the other hand the approach via inspections, requiring no science, no new methods, only common sense. The latter turns out to be effective. How is it possible that the effective, easy, and obvious was overlooked in favour of the difficult, expensive, and impotent? Such a perverse preference can only be explained by a mindset which turns the obvious into the opposite. If this is indeed so, the mindset must influence other affairs than those related to software development. I will here quote evidence that this is indeed the case, that indeed the mindset is so pervasive as to have been identified elsewhere well before the first attempts at formal verification of programs, that belated identification of the mindset in computing brands this field as a backwater in the world of ideas.

The mindset is called “modernism”; its sane alternative goes by the name of rhetoric. The onetime success of software inspections was, in computing, a shortlived triumph of rhetoric over its modernist alternative.

In the remainder of this essay I will sketch various aspects of modernism. Of rhetoric I will say little more than the obvious: that “specious eloquence” is not its primary meaning. In the Humanities side of C.P. Snow’s “Two Cultures” it is not necessary to make this point. On the other side it very much needs to be made. Rhetoric, then, is the art of persuasion. This is the case independently of whether the motive for persuasion is honourable. The primary tool of rhetoric is natural language. In certain specialized and limited contexts graphs, tables, and algebraic expressions are supporting tools of rhetoric. It is the modernist delusion to believe that such specialized and limited contexts are strongholds from which the entire range of rational discourse is to be conquered.

My starting point has been a paper by D. McCloskey entitled “The Rhetoric of Economics” (now a book). McCloskey argues that rhetoric should be the main intellectual tool of economists, but that this central position has been usurped by modernism, the complex of attitudes and techniques that McCloskey introduces in the following way:

(It) is an amalgam of logical positivism, behaviourism, operationalism, and the hypothetico-deductive model of science. Its leading idea is that all sure knowledge is modeled on the early 20th century’s understanding of certain pieces of 19th-century physics. To emphasize its pervasiveness in modern thinking well beyond scholarship it is best labeled simply “modernism,” that is, the notion … that we know only what we cannot doubt and cannot really know what we can merely assent to.

By now modernism has achieved a venerable old age. Here are some modernist quotes; one each from the 17th, 18th, 19th, and 20th centuries.

Leibniz (1646 — 1716) envisioned a generalized mathematics, by which thinking could be replaced by calculation:

If we had it, we should be able to reason in metaphysics and morals in much the same way as in geometry and analysis… If controversies were to arise, there would be no more need of disputation between two philosophers than between two accountants^[*]. For it would suffice to take their pencils in their hands, to sit down to their slates, and to say to each other (with a friend as witness, if they liked): let us calculate.

Moving to the 18th century, we have the following from David Hume:

When we run over libraries, persuaded of these [modernist] principles, what havoc must we make? If we take in our hand any volume — of divinity or school metaphysics, for instance — let us ask, Does it contain any abstract reasoning concerning quantity or number? No. Does it contain any experimental reasoning concerning matter of fact and existence? No. Commit it then to the flames, for it can contain nothing but sophistry and illusion.

Lord Kelvin:

When you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.

Although for now modernists have the rostrum, I can’t help inserting at this point the following postmodern retort: “Yes, and when you can express it in numbers, your knowledge is of a meagre and unsatisfactory kind.”

As representative of the 20th century, I select Wittgenstein whowrote:

Anything that can be said at all, can be said clearly. And of what cannot be said clearly, one must not speak.

Again a postmodernist retort (Michael Polyani in 1962): the methodology of modernism sets up “quixotic standards of valid meaning which, if rigorously practiced, would reduce us all to voluntary imbecility.

McCloskey was concerned with the ill effects of modernism in economics. Susanne Langer (“Philosophy in a New Key”, 1941) makes a similar observation for psychology:

Psychologists have probably spent almost as much time and type avowing their empiricism, their factual premises, their experimental techniques, as recording experiments and making general inductions.

A few pages later:

The physicists’ scheme, so faithfully emulated by generations of psychologists, epistemologists, and aestheticians, is probably blocking their progress, defeating possible insights by its prejudicial force. The scheme is not false … but it is bootless for the study of mental phenomena. … Instead of a method, it inspires a militant methodology.

And not only psychologists, epistemologists, and aestheticians suffer self-inflicted debilitation by adopting modernist trappings. This can also be said of sociologists, management scientists, and computer scientists.

The modernist urge to derive respectability from the formal, the abstract, and the numerical has given matrix algebra, especially of the computational variety, an unwarranted prominence in management science and psychology (just as there exists Mathematical Physics, there is Mathematical Psychology; see e.g. “Handbook of Mathematical Psychology”, three volumes, 1965). As a result, some papers are published in the journals of one these disciplines that could as well have appeared in any of the other ones. Researchers are welcomed to their university faculties mainly on the strength of their expertise in matrix algebra. Their prime allegiance is to matrix algebra rather than to the discipline of their department.

Mathematics plays a crucial role in the understanding of the tension between modernism and rhetoric. On the one hand the modernist mind has seen mathematics as an example to be followed: surely here is the ultimate in precise and secure knowledge, an example to be emulated by all scholarship. On the other hand, it has escaped the modernist that mathematics is still done by old-fashioned rhetoric. Where the modernist psychologist is a heavy user of computers (for computation), the mathematician has no such need. Hilbert’s program, the quintessence of modernism, has never been followed in mathematics. However, it is the precursor of the program verification enterprise in computer science.

This is not to say that mathematics and program verification should only be conducted in natural language. Algebraic notation, introduced centuries ago, is now very much accepted in the rich, unmodernist rhetoric of mathematics. It is important to realize that algebraic notation can be used as extension to natural language to clarify (to the (human) mind) what would be obscure otherwise. But such use need not be part of a formal method. Algebraic notation can be used in a rhetoric that is both rigorous and informal. At one time, then, a successful shift has occurred that incorporated algebraic notation into the language of rhetoric.

On the one hand, the modernist excesses of the recent times must be rectified. On the other hand this must not imply that rhetoric should forever be shielded from further enrichment by modest incorporation of additional algebraic notation. We must keep in mind that modest steps are called for: so far anything beyond English enriched by formulas, graphs, and tables has proved to be sterile. And the additions should be made by those who appreciate the power of traditional rhetoric in mathematics. This disqualifies for example E.W.Dijkstra (see his note EWD1012), who denounces this rhetoric as a manifestation of the spirit of the medieval guilds: calculated to maintain spurious barriers preventing entry by the laiety. Apparently Dijkstra believed that it is territorialism that prevents the process of mathematical discovery from being formalized and automated without loss of power.

To return to the success of rhetoric in computing in the form of the code inspection, I note that in its reported form, inspections are traditional rhetoric, unaided by any symbolic logic. I believe that an informal version of some program verification techniques, such as Floyd’s method of assertions, can make inspections more effective. My guess is that the optimum consists of including assertions into code, which consist of English fortified by logic formulas. This is far from a formal system and much like the traditional rhetoric of mathematics.

I can’t leave well enough alone. The most recent manifestation of this problem is the following program from Kernighan and Ritchie (“The C Programming Language”), which is perfectly OK code for converting a number to the equivalent decimal numeral.

void itoa(int n, char s[])
{
    int i, sign;

    if ((sign = n) < 0)  /* record sign */
        n = -n;          /* make n positive */
    i = 0;
    do {       /* generate digits in reverse order */
        s[i++] = n % 10 + '0';   /* get next digit */
    } while ((n /= 10) > 0);     /* delete it */
    if (sign < 0)
        s[i++] = '-';
    s[i] = '';
    reverse(s);
} 

This requires something presumably like the function below, which I list so that lengths of code can be compared.

void reverse(char s[]) {
  int i = 0, j = 0;
  while (s[j++] != '')
    ;
  j -= 2;
  while (i < j) {
    char temp = s[i];
    s[i] = s[j]; s[j] = temp;
    i++; j--;
  }
}

What is it that I don’t like about it? For one, it is left up to the user to supply a sufficiently long string. And wouldn’t it be more natural for a routine with one input and one output to deliver that output as the function value? As in

char* itoa(int n)

?

Then, the conversion is such that the digits come out in the wrong order, so that in a separate pass they have to be reversed. Moreover, reverse is called in such a way that, it seems to me, it has to make two passes over the string.

These thoughts need to be quelled right away: OK, we have to guess at a suitable length for the numeral, but one can hardly guess very wrong: after all, an int can only have a very limited range of sizes. And, as for the digits coming out in the wrong order, isn’t it a convenient order and isn’t it a good problem decomposition to have them come out in the convenient order first and use the easiest possible piece of code to obtain the desired order?

Yes, indeed I should have left it alone. One should use the most un-subtle solution to a simple problem, and get on with the job. But just before conceding the point, I would like to point out a subtlety in this supposedly straightforward code. The convention for numerals is not to have leading zeros. This rule has an exception: the numeral for the integer zero does have a leading zero, otherwise it would be the empty string.

The empty string is the mathematically correct representation of the number zero. The problem is that we don’t have a symbol for the empty string. Through the recent centuries the public learned to accept the mathematical subtlety of a digit to represent zero (“nothing”), but unfortunately stopped short of accepting the analogous need for a symbol for an empty string. When you write the usual “while … do” in itoa you will get the empty string for the integer zero. Only by using the unusual “do … while” do you get the required exception. In the interest of straightforward coding, I would prefer to catch the exceptional case up front and let the rest of the code follow what is natural from a mathematical point of view.

I rewrote itoa to accommodate the above gripes, knowing that I can’t charge the time so spent to the Project. The result may strike you as unnecessary show-off: harder thinking than necessary for a task that can be as simple as demonstrated by Kernighan and Ritchie. Compare it to a flip-kick in soccer: necessary in tight situations, with success meriting a big applause. But if a player would do one without due necessity, he would deserve a Boo. What I’m doing in the sequel is like an uncalled-for flip-kick. It is educational (ambitious soccer players practice flip-kicks), so get students to write it. Or add it to your repertoire for an in-depth interview.
In his comment of March 9, Michael Levy suggested a preliminary pass to find out how many digits the numeral has, then allocate, then the digits can be written from right to left, as in the following:

char* mrl_itoa(int n) {
  char* result;
  if (n == 0) {
    result = calloc(2,sizeof(char));
    result[0] = '0';
    return result;
  }
  int neg; if (neg = n0
  int m = n, numDgts = 0, k;
  do {m = m/10; numDgts++;} while (m > 0);
  // numDgts is the number of digits of positive n
  result =
    calloc(k = numDgts  // for digits of n
             + neg       // for sign, if present
             + 1         // for terminating null
           , sizeof(char));
  m = n;
  for(k = k-2; k >= neg; k--) {
    result[k] = '0' + m%10; m = m/10;
  }
  if (neg) result[0] = '-';
  return result;
}

But do we really have to make two passes? First, why do the digits seem to want to come out in the wrong order? The answer is another question: Why do we think that iteration is the only way to do repetition? There is another way: recursion. It is has the advantage in this case that we do have control over the order in which the digits are printed.

To make them come out the wrong way, write

void itoa(int n) {
  if (n == 0) return;
  print(n%10); itoa(n/10);
}

This reads like: to print out the digits in reverse order, print the last followed by the remaining digits in reverse order. Now it’s obvious what to do out about it:

void mitoa(int n) {
  if (n == 0) return;  mitoa(n/10); print(n%10);
}

This reads like: to print out the digits in order, print in order all digits except the last followed by the last digit. The attraction of this scheme is that the digits are immediately deposited in the right place.

Let us now take care of catching the digits in a string instead of printing them as soon as they are generated. mitoa() has the additional advantage that at the end of the recursion it is known how many digits there are. At that point we can allocate the right number of characters. We modify the function to return the location in the string where the next character is to be deposited. Or, rather, we define an auxiliary function to do this.

int mitoa1(int n, int depth, char** s) {
// write numeral from s[depth] onward
// return index for next digit
  if (n == 0) {
    *s = (char *) malloc(sizeof(char)*(depth+1));
    *(*s + depth) = '';
    return 0;
  }
  int i = mitoa1(n/10, ++depth, s);
  *(*s + i) = '0' + n%10;
  return i+1;
}
char* mitoa(int n) {
  char* t;
  mitoa1(n, 0, &t);
  return t;
}

Look: nice and short; no reversing. So far the pretty part. Now for the uglies: the minus sign in case of a negative number. Here I admit that writing the digits first in reverse order has an advantage: you can just stick the minus sign on at the end, before the reversal. In my approach the minus sign has to be deposited once and for all in the right place. So, for the first character I need to distinguish between printing a negative or a positive numeral. Another special case.

int mitoa1(int neg, int n, int depth, char** s) {
// write numeral from s[depth] onward
// return index for next digit
  if (n == 0) {
    if (neg) {
      *s = (char *) malloc(sizeof(char)*(depth+2));
      *(*s + depth+1) = '';
      return 1; // leave room for minus sign at index 0
    } else {
      *s = (char *) malloc(sizeof(char)*(depth+1));
      *(*s + depth) = '';
      return 0;
    }
  }
  int i = mitoa1(neg, n/10, ++depth, s);
  *(*s + i) = '0' + n%10;
  return i+1;
}
char* mitoa(int n) {  char* t;
// will point to the digits of n
  if (n == 0) { // the special case
    t = (char*) malloc(sizeof(char) * 2);
    t[0] = '0'; t[1] = '';
    return t;
  }   // the general case
  int neg = (n<0)?1:0;
  if (neg) n = -n;
  mitoa1(neg, n, 0, &t);
  if (neg) *t = '-';
  return t;
}

The following improvements have been implemented.

1. The storage allocation is done by the function rather than by the user.
2. The function gives its result as the returned value rather than as
   a modification to an output parameter.
3. Exactly the right number of characters is allocated.
4. The computed digits are put down in the right place once and for all.

But is this really a one-pass algorithm? It relies on finding the length of the numeral on the way down in the recursion and on writing the digits on the way back up. And a function call per digit may be more expensive than Michael Levy’s division per digit for finding the length.

Consider this:

1. Software reliability is rapidly becoming more
   important.
2. Software development technology is inadequate and
   is only slowly improving.

The question arises:

Is disaster inevitable?

This has been debated for some time in the computer
science community. Recently it has begun to attract
attention from the general public.
For example a newspaper ran a story entitled: “A Lemon Law for
Software?” Here the author observed:

If Microsoft were liable for product defects the same
way automobile manufacturers are, then it would have
been driven out of business a long time ago.

As we all know, this has not happened. And this is only
because the law exempts the software industry from
product liability.

The question I’m now asking is: Should it be exempted in
this fashion?

Any suggestion that it is time for the software industry
to grow up and accept responsibility for its products is
greeted by an indignant chorus asserting that software is
special and that not recognizing it as such will stifle
innovation.

Clearly Microsoft stands to lose most in a change from
the status quo. But the rest of industry, including those
who love to hate Microsoft, do not dare to contemplate a
world where creators of software are responsible,
seriously responsible, for their products.

While in industry there is agreement on this topic,
academia, in so far as it has an opinion at all, is
divided. Some argue that software is special in such a
way that the current unreliability of software is
inevitable. They argue that there is no prospect of
drastic improvement. I will call this camp “The Real
World”. Their most eloquent, and most widely quoted,
spokesman is Frederick P. Brooks, Jr., in his paper “No
Silver Bullet”.

Others argue that techniques for producing reliable
software are known, that these are practical, and that
some of them have been known for decades. Whenever one
brings these to the attention of industry, one is
dismissed as being sadly out of touch with reality. Hence
I will denote the proponents of the known techniques for
producing reliable software as “The Ivory Tower”.

In this introduction I have sketched the problem,
which is:

1. Software reliability is rapidly becoming more
   important.
2. Software development technology is inadequate and
   is only slowly improving.

I have also sketched the main points of view, and called
them “The Real World” and “The Ivory Tower”. In this
lecture I will elaborate further on these two opposing
positions, starting with the real-world people, and
continuing with those who inhabit the Ivory Tower.
Although I will avoid technical matters, few such
concepts need a brief review, which I will do next.

Intermezzo on Program Reliability

Let me remind you that no law concerning the physical
world is known with certainty. For example, not even
Newton’s laws. An experiment can disprove them, but no
amount of experimentation can prove them. Philosophers of
science have been familiar with this ever since Whewell
in the nineteenth century.

With the correctness of programs we are in a similar
position: a test can disprove it, but no amount of
testing can prove it.

But programs differ from laws of nature in an interesting
way: it is sometimes possible to model the correctness of
the program by a mathematical theorem, which is
susceptible to proof. Even in this favourable case,
certainty eludes us: the theorem is only a model of the
program acting on the physical world; moreover,
mathematical proofs may have errors in it. It is tempting
to have the proof done by computer, but then how do you
prove that the theorem-proving program is correct?

Now, it is the case that proving a program correct
enormously enhances its reliability. Although testing is
much weaker, it can also enormously enhance reliability.
In the case of testing it needs to be emphasized that it
needs to be done right, which is difficult. Of course,
the same holds for proving a program correct, but somehow
that does not need emphasis: every programmer thinks he
can test; rare is the programmer believing she can prove.

Because certainty in this matter is not possible, I will
talk about program reliability instead of correctness.
Correctness is a true-or-false concept. Reliability
covers a whole spectrum. On the scale that I have in
mind, Microsoft Windows and the typical applications
running on it are not reliable. An operating system where
security patches are needed on a recurring basis, is
another example of unreliability.

The Real-World Brigade

After this interlude about reliability versus correctness
and testing versus proving, I continue with what I call
the “real-world brigade”, which encompasses almost all of
industry and most of academia. Frederick Brooks wrote a
paper called “No Silver Bullet” that expresses the world
view of the Real-World Brigade so eloquently that he has
become, whether he likes it or not, the ideologue of the
status quo.

Let us first see where that title comes from: “No Silver
Bullet”. I quote from the beginning of the paper:

Of all the monsters that fill the nightmares of our
folklore, none terrify more than werewolves, because
they transform unexpectedly from the familiar into
horrors. For these, one seeks bullets of silver that
can magically lay them to rest.
The familiar software project, at least as seen by the
non-technical manager, has something of this
character; it is usually innocent and straightforward,
but is capable of becoming a monster of missed
schedules, blown budgets, and flawed products. So we
hear desperate cries for a silver bullet–something to
make software costs drop as rapidly as computer
hardware costs do.

Brooks sees a long parade of academics touting methods to
supposedly make software costs drop rapidly. He reviews a
list that I will recite in a moment. Before I do so, you
need to know that the paper was presented in 1986.

Here are, then the purported “silver bullets”: Ada and
other high-level languages, object-oriented programming,
artificial intelligence, expert systems, automatic
programming, graphical programming, program verification,
environments and tools, workstations.

All of these, Brooks point out, only address the
representation of the conceptual construct of the
software system. The difficulties of representing the
construct are only an accidental part of the difficulties
of software development. The essence of the software
system is the conceptual construct itself.

At the opposite of the accidental, one has the essential.
What is then, according to Brooks the essence? He says
this.

Let us consider the inherent properties of this
irreducible essence of modern software systems:
complexity, conformity, changeability, and
invisibility.

As I said, Brooks is eloquent.

He has much to say about the first of the Ferocious
Four: complexity. I will just quote:

The complexity of software is an essential property,
not merely an “accidental” one. Hence, descriptions of a
software entity that abstract away its complexity
necessarily abstract away its essence.

By “conformity” Brooks means that, as software is
supposedly infinitely adaptable, it must span the vast
chasm between unadaptable humans and unadaptable
hardware.

By “changeability” Brooks means that, as any change in
software can be effected by a mere change in text, all
desire for change are to be met as a matter of course by
software.

By “invisibility” Brooks means that neither text nor
graphics are an adequate representation of the conceptual
constructs that grow rampantly in response to marketing
demands and unanticipated programming difficulties.

Such are, according to Brooks, the inherent properties of
the irreducible essence of modern software systems.

A voice from the opposition

In 1972 Edsger W. Dijkstra received the Turing award from
the Association for Computing Machinery. On the occasion
he delivered an address entitled “The Humble Programmer”.
The content has been, as far as I can tell, completely
ignored by the world. In defence of this it can be said
that a prediction made in this paper has failed to come
true.

In fact, Dijkstra did not present it as a prediction. He
said:

Let me sketch for you one of the possible futures. At
first sight, this vision of programming in perhaps
already the near future may strike you as utterly
fantastic. Let me therefore also add the
considerations that might lead one to the conclusion
that that vision could be a very real possibility.

The vision is that, well before the seventies have run
to completion, we shall be able to design and
implement the kind of systems that are now straining
our programming ability at the expense of only a few
percent in man-years of what they cost us now, and
that besides that, these systems will be virtually
free of bugs. These two improvements go hand in hand.
In the latter respect software seems to be different
from many other proudcts, where as a rule higher
quality implies higher price. Those who want really
reliable software will discover that they must find
means of avoiding the majority of bugs to start with,
and as a result the programming process will become
cheaper. If you want more effective programmers, you
will discover that they should not waste their time
debugging — they should not introduce bugs to start
with. In other words, both goals point to the same
change.

Dijkstra went on to argue that this future was indeed
possible. He recognized three necessary conditions. The
third of these conditions is especially relevant for
tonight’s topic: is the revolution sketched by him
technically feasible?

He advances a number of arguments that it is. More
important than the arguments themselves is his pre-amble
to them:

I now suggest that we confine ourselves to the design
and implementation of intellectually manageable
programs. If someone fears that this restriction is so
severe that we connot live with it, I can reassure
him: the class of intellectually manageable programs
is still sufficiently rich to contain many very
realistic programs for any problem capable of
algorithmic solution.

Here it is: the irreconcilable juxtaposition. Brooks
claiming that complexity is one of the inherent
properties of the irreducible essence of modern software
systems. Dijkstra claiming that we stand to lose nothing
if we restrict ourselves to intellectually manageable
programs. This is the reason why Dijkstra gave as title
to his speech: “The Humble Programmer”.

Thirty-five years have gone by since Dijkstra’s speech. It was halfway
that period when Brooks so tellingly crystallized the conventional
wisdom on this matter. Now we hear suggestions to stop the exemption
of product liability enjoyed by software firms. This is a good time to
add an observation that Dijkstra could have made.

The observation is this. One of the foundational dogmas of software
“engineering” is that one starts out with a requirements specification
that is independent of any implementation consideration. To do
otherwise would be to violate the sequence according to which
implementation only comes after design, which comes after the
requirements process. To do otherwise is to invite chaos. The actual
artifact eventually emerging is supposed to be a complete surprise.

In all this preoccupation with process, intellectual manageability is
never a consideration. It may be very hard to predict from a
requirements specification whether the subsequent design and
implementation is going to be intellectually manageable.

Tellingly, the most successful area of software development is ignored
by software engineering, and it ignores software engineering:
compilers. This is one kind of software that often is intellectually
manageable and where a new project consists of modifying an existing
intellectually manageable system. Here is one kind of project without
“… missed schedules, blown budgets, and flawed products”, as Brooks
so aptly describes some other kinds of project.

And this is exactly how innovation proceeds in engineering. I mean
real engineering, not software development. Real engineers are liable
for failures in their products. Dijkstra’s proposal to restrict
consideration to intellectually manageable designs, which is
considered so utterly unrealistic in software, is utterly obvious to
engineers. So obvious that it may never even have been stated.

Of course, there is plenty of scope for unmanageable complexity in
engineering. There are plenty of ideas for new designs, new materials,
new production methods, new computation methods, … Unmanageable
complexity is not the prerogative of software.

But an engineer knows he’s going to be personally liable if he takes
but one small step beyond what’s intellectually manageable. Compared
to software “engineers”, engineers are humble people: they know their
limitations. They have to.

What then to do about Brooks’s typical software development project
that

…is usually innocent and straightforward, but is
capable of becoming a monster of missed schedules,
blown budgets, and flawed products?

The silver bullet that Brooks says does not exist, is a
humble programmer, who keeps complexity within the bounds
of manageability. It is a software engineer who is an
engineer.

There is an old song about software engineering. I thought it had died until the other day I came across a post by Jack Ganssle (Embedded.com; 05/12/08). Under the title “Programmer or Engineer?” he argues that the word “programmer” should be banned and that embedded applications should by built by electrical or computer engineers.

It may well be that engineers are satisfactory for the “embedded applications” that Mr Ganssle has in mind. But another category of people is needed to meet the greater challenges posed by computers. Neither of the existing categories: “engineer” nor “computer scientist” can be relied on to fit the bill.

The existing categories are not suitable

The unsuitability of computer scientists is charmingly documented by the late Alan Perlis:

I think that it’s extraordinarily important that we in computer science keep fun in computing. When it started out, it was an awful lot of fun. Of course, the paying customers got shafted every now and then, and after a while we began to take their complaints seriously. We began to feel as if we really were responsible for the successful, error-free perfect use of these machines. I don’t think we are. I think we’re responsible for stretching them, setting them off in new directions, and keeping fun in the house. I hope we don’t become missionaries. Don’t feel as if you’re Bible salesmen. The world has too many of these already. What you know about computing other people will learn. Don’t feel as if the key to successful computing is only in your hands. What’s in your hands, I think and hope is intelligence: the ability to see the machine as more than when you were first led up to it, that you can make it more.

My belief that engineers don’t do any better, though for different reasons, is based on observations of a few of my friends who have transcended the distinction between engineer and computer scientist.

A proposal by Parnas

Ganssle proposed to qualify only electrical or computer engineers for the programming of embedded systems. Things were not so simple for David Parnas, who pondered the unsuitability of computer scientists a long time ago. He was under no illusions about the suitability of those designated “electrical engineer” or “computer engineer”. Accordingly he advocated a new branch of engineering.

Parnas’s proposal was embraced in many quarters because it responded to the overwhelming feeling that something urgently needed to be done. Yet the situation may not be as clear-cut as Parnas presents it. He pointed out that today’s students’ careers could last four decades and stated:

“We must identify the fundamentals that will be valid and useful over that period and emphasize those principles in the lectures.”

We must, we must, for sure. But can we?

Apparently, the idea is that it is better to have fundamentals of something than no fundamentals. In fact we need capable and responsible programmers. What is proposed is a curriculum that differs from computer science by, for example, dropping courses on compilers and operating systems, and, for example, including General Chemistry for Engineers and Introduction to the Structure and Properties of Engineering Materials. The latter presumably have the virtue of Fundamentality, a unary property that makes it improper to ask: Fundamental to what?

Engineering from craft to profession

To see what is wrong with Parnas’s proposal, it helps to review the history of engineering. A brief and enlightening account can be found in John Allen’s paper “Whither Software Engineering?”, to be published soon.

We take it for granted that the established branches of engineering, Civil, Mechanical, Chemical, and Electrical, are based on science: physics and chemistry, and that these rely on nontrivial mathematics, mainly calculus. These are Fundamentals for the established branches of engineering. Hence the (now) established practice of having calculus and science courses as a prominent part in the first year of engineering programs.

However, this has not always been the case: it is the result of a long transition in engineering from shop culture to school culture. The transition began in France in the 18th century, spread to Germany in the early 19th, and to England and the US in the late 19th century.

In the early 19th century it was appropriate to call the builders of the early steam engines “engineers”, even though their work was not based on science. Thermodynamics was not just less developed than it is now: it did not exist. Engineering was in the stage that Allen calls “shop culture”.

Fast forward to software development in the year 2008: the underlying science does not exist; software development is in the craft stage. A possible transition into the professional stage is still in the future, if anywhere.

Thus the very term software engineering is misleading. It is OK to consider the practitioners of two centuries ago as engineers because all engineering was in the craft stage. But now that engineering is based on science, the very term “engineering” implies a discipline of that level. It is not OK to consider graduates of the proposed Parnas curriculum as engineers. They may have taken courses labelled as Fundamental, but they are not fundamental to software development. Or they may have been trained as electrical engineers, just as they may happen to have been trained as computer scientists or physicists. But as software developers they necessarily are craftsmen because a scientific basis for software development is lacking.

Cargo cult science

Parnas’s proposal reminds of the late Richard Feynman’s CalTech 1974 commencement address. Feynman explains why, even though they perfectly emulate science, certain studies in psychology are not. All the right moves are there: Null Hypotheses, Type I and Type II errors,…, the works. But though the form of these studies is perfect, the exercise is vacuous. He called this Cargo Cult Science, after the phenomenon observed shortly after the second world war on some of the islands in the Pacific. From Feynman’s address:

During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas — he’s the controller — and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No planes land.

We are in a similar situation. Over there’s the Paradise of Engineering. Whether it’s Civil, Mechanical, Chemical, Electrical, by and large those engineers are successful in delivering safe and satisfactory installations and products on time and within budget. Over here we are in software land where the most egregious failures and cost overruns are of the order of the day. Of course we want the same successes to happen Over Here. So we arrange everything just like it is Over There. Get every little detail right, the General Chemistry for Engineers and the Introduction to the Structure and Properties of Engineering Materials, the works. We have no idea how, but somehow the power of sympathetic magic will deliver the same successes Over Here.

Parnas’s proposal for a new engineering discipline, as the latest member in the sequence of civil, mechanical, mining, and electrical engineering, was an exercise in cargo cult engineering.

A new category is needed

The prime example of the type that we very much need is the late Edsger Dijkstra. He performed engineering feats of the highest order that no engineer, nor anybody else, had achieved: the lowel-level I/O software for the Electrologica X-1 computer, the first compiler for Algol 60, and the THE operating system.

Dijkstra was conscious of being of a new breed, and he chose “programmer” for its name. He considered programming as a discipline that required the highest intellectual level. This claim has to be taken seriously from someone who was educated as a theoretical physicistat a good university.

Compare this with the sense that Mr Ganssle assigns to the word “programmer”:

A programmer is one who writes programs. That’s subtly like “coder,” a person who cranks code out all day. Coders, like technicians and janitors are vitally needed employees. But they can be devastating to software projects.

For Mr Ganssle the remedy is simple, just avoid “programmers” and make sure you hire certified engineers:

Most of us are engineers: software engineers, firmware engineers, or hardware/software engineers, no matter what degree we earned. In fact, in a recent Embedded poll only 13% of respondents claimed only a computer science degree; 80% have a sheepskin with an engineering title (EE or CE).

Parnas’s proposal revisited

Logic compels me to conclude that software development, being in the craft stage, should be taught according to the guild system. But, being a university man, I like to believe that these institutions have something to contribute.

Does this mean that computer science departments can supply the programmers we need? No, there faculty are hired and maintained on the basis of scholarly publications. A great programmer would not be accepted as such on the faculty. Dijkstra was welcomed in a Computer Science department as a celebrity computer person, not as a computer scientist.

Does this mean that universities cannot help in fostering programmers? Not at all. Some departments have suitably different standards: they welcome to their faculty performers rather than generators of publications in refereed scholarly journals. Departments of Music are an example.

There is another way in which universities can help. In the 1950s the first programmers were recruited on no other basis than being a promising oddball. One of the prerequisites for being counted in this category was a nontrivial college degree (Mark Halpern — English literature, Edsger Dijkstra — theoretical physics, Tony Hoare — classics). This in itself ensured, though perhaps not exceptional intelligence, at least not a debilitating lack of it. But in these particular cases, exceptional talent was recruited. So this is something that universities can do as help toward great programmers: organize demanding programs wherever they can, independently of computers or programming.

Of course we can be less hit-and-miss than this, now that we at least have the concept of “programmer”, even when we do not have a clear idea of how to help to bring them into being. For example, Ars Digita University, which existed from 2000 to 2001, taught a ten-month course in programming (not “software development”, nor “software engineering”) to graduates with solid college degrees, ideally without exposure to computer science or software development. In each of the ten months of the course, students studied one of ten books (typical representatives: Abelson and Sussman, Patterson and Hennessey, Cormen/Rivest/Leiserson). There was lots of lab work. It was an intensive course.

(from “The ALP Newsletter”, vol. 17, no. 1, Feb. 2004)

While it is well-known that it is hard to know anything about the future, it is less widely realized that it is also difficult to understand the present. It is therefore paradoxical that it sometimes helps to understand the present by means of a fictitious future history. In this essay I am going to exploit this paradox.

I place the viewpoint well away into the future. Again this may seem paradoxical, because in computing everything is supposed to happen fast. Not everybody thinks so. Paul Graham has embarked on a project that he calls “The Hundred Year Language” , pointing out that it is only hardware that changes fast. Programming languages change slowly. This is because they are part of culture; they reflect how human minds tackle problems. Hence Graham’s “Hundred Year Language”.

It so happens that Graham is concerned with Lisp. This is not the only paradigm in need of the long-term perspective. Accordingly, I imagine a history of logic programming as it may be written in the 2020s.

As I’m already exercising your indulgence with speculation, I will not attempt the fictitious future history itself, but instead bring out the salient points in an equally fictitious review of this nonexistent
history.

A Critical Review of Karl Senner’s “History of Logic Programming”

When the first edition of Senner’s history came out in 2020, it was widely praised for its compelling view of the development over many decades of logic programming. Reviewers praised it for its broad perspective, but deplored its lack of historical detail. Since then several collections of papers have made their way from estates via the auctions to various libraries. Senner has taken this opportunity to incorporate these recent findings in a new edition.

What has not changed in the new edition, and this is what I take issue with, is that Senner presents the development of logic programming as a relentless forward march towards an inevitable outcome. Of course, it is true that not a decade went by without some of the building blocks being fashioned that we take for granted as part of the majestic edifice that now dominates the landscape of programming.

The way Senner tells it, and his readers love him for it, is that there was a continuous forward movement. This certainly makes for a good story. But it does not recognize the fact that the received view of history is only as recent as the Tens. What I’m now asking the reader to imagine, Senner’s book in hand, what the situation of logic programming must have looked like in the mid Zeroes. I think you will agree that Senner’s story of a relentless march forward is an artifact of 20/20 hindsight.

Let us imagine it is 2005, to pick a representative year in the mid Zeroes. The great contributions of the Nineties were known; they were even widely celebrated. But were they known, in 2005, as contributions to logic programming? Senner, of course, does not say so. But he makes it easy to overlook the fact that in 2005 nobody seemed to recognize as such the contributions of the Nineties to logic programming. Let us consider three of these contributions here: the XML movement, the compilation/execution model based on virtual machines, and constraint programming.

XML

The XML movement, which arose from the World-Wide Web in the mid Nineties, can be characterized as exploiting the tree as universal datastructure. We now view this as a useful further stage in a development that started in the early Seventies. But if you would have whispered “Colmerauer” into the ear of an XML devotee, you would have met with a blank stare. Following it up with a heavy hint like “Prolog” wouldn’t change anything.

Nor were the few remaining logic programmers excited by the XML movement. They regarded it as part of the big, bad, ugly outside world that had robbed them of their lawful place in the limelight. From their point of view, XML and all its works was only something to reluctantly accommodate with yet another ad hoc interface, not something to be embraced as a tree technology to serve as the foundation of a new version of Prolog. From our present vantage point it is difficult to imagine how compartmentalized computing was at the time.

From our vantage point, 2005 was indeed a year of delicious ironies. XSLT was already gathering a following, with its adherents discovering that one could do all kinds of computational tasks as transformations on (XML) trees. Working from the other side, Yeow, working with WAM experts Horspool and Levy, had shown in a 2002 paper that parts of the WAM design could be used for a much faster XML parser. Yet the penny had not dropped.

Execution via virtual machine

Another innovation in which the Prolog of the Eighties was hopelessly far ahead of its time was execution via a virtual machine. At the time, Prolog advocates pointed out that when Prolog returns the answer to the user in 100 milliseconds, it is irrelevant that a lower-level implementation executes in 10 milliseconds. To no avail. The very fact that Prolog was “interpreted” marked it as unfit for the “real world”.

One of the great contributions of the Nineties was the “real world” discovering the advantages of the compilation/execution model of WAM based Prolog in the form of the Java Virtual Machine. Perhaps in some quarters it was still somewhat suspect, SUN being a notoriously innovative company. The last doubts disappeared when this model was adopted by a company far above any suspicion of innovativity.

Constraint Programming

The third important development of the Nineties is constraint programming. Here especially Senner fails to give a sense of the many fumbling steps that were taken towards the software architecture of logic programming that we now take for granted. Indeed, we now take it so much for granted that a brief tutorial on the basics may be needed.

In the mid Seventies, it was customary to distinguish “Prolog” from “Pure Prolog”. The latter was almost the same as the use of SLD resolution with Horn clause programs. In Pure Prolog one can perform transformations of trees containing symbolic values only. Counting and arithmetic have to be simulated by such symbolic computation. To allow access to the processor’s arithmetic, Colmerauer and Roussel had already added extra-logical features to Pure Prolog. Though this resulted in a practical language, it made it difficult to characterize the class of program-query pairs that yield logical consequences.

Colmerauer himself would probably not have agreed to this analysis of Prolog. For him Prolog was not part of logic programming. For him Pure Prolog was not an inviolable given just because it was resolution theorem-proving. He clarified his position by showing that Pure Prolog itself should be decomposed by regarding unification as constraint-solving over terms. The immediate use of this was to clarify the controversy over the occurrence check in unification. Colmerauer showed that constraint solving over finite terms (which are the constituents of Herbrand universes) corresponds to unification with occurrence check. He presented a unification algorithm that is more efficient by omitting the occurrence check and showed that this algorithm can be viewed as constraint solving over rational terms.

Let us denote by CP(S) constraint programming over a structure S. CP(S) embraces not only a structure in the sense of model theory (consisting of a carrier, constant elements, constants, relation symbols, function symbols), but also a system of efficiently representable sets of values and efficiently computable contraction operators for the constraints.

A good way to illustrate the distinction between CP(S) and S is to consider the structure R of the reals. Constraint programming over R adds to R a system of selecting certain sets of reals as efficiently representable in the form of intervals. It also adds contraction operators for these intervals corresponding to constraints over the reals such as the ternary sum constraint x + y = z and the ternary product constraint x * y = z.

Similarly the structure of ground Herbrand terms can be enriched by adding a system for efficiently representing certain sets of ground terms. The preferred system is to use a term to represent the set of its variable-free instances. This efficiently represents a sufficiently rich repertoire of sets of ground terms. The constraint is equality and the efficiently computable contraction operator is unification. This gives CP(FT). The same method, but starting with the structure of rational ground terms gives CP(RT).

The deconstruction of Pure Prolog suggested by Colmerauer’s work would now be written as:

Schematic Prolog + CP(FT).

Colmerauer was more interested in

Schematic Prolog + CP(RT).

and was not concerned with the fact that it is not a form of resolution theorem proving.

Colmerauer’s work in the late Seventies was concerned with justifying unification without the occurrence check. The deconstruction of Pure Prolog is implied. It was made explicit by Jaffar and Lassez in their CLP Scheme. It is for this reason that the core that remains of Pure Prolog after separating unification is called “Schematic Prolog”. The CLP Scheme explicitly allows the addition of any number of constraint programming structures to Schematic Prolog. For example,

Schematic Prolog + CP(FT) + CP(R)

is an interesting Prolog, as it uses the techniques of interval arithmetic to obtain a logically sound method for the approximate solution of equations over the reals. The answer constraints are in the form of membership of intervals with floating point numbers as bounds. This soundness of these answers is not affected by the inevitable rounding errors that occur in the computation of these answers. By incorporating machine arithmetic into Prolog in this way, an important motivation for the extra-logical features of the original Prolog had disappeared.

By 2005 several important CP structures were known. We already mentioned CP(FT), CP(RT), and CP(R). Finite-domain constraints fit in this framework. The representable sets include the entire powerset. The constraints are equality, disequality, and alldiff. We call the resulting system CP(FD). Boolean constraints were well known. Although constraint programming over integers had played an important role since CHIP, it was still not soundly implemented because integer overflow invalidated the result if it occurred. But of course, CP(R) plus the unary integer constraint in principle made CP(Z) available.

Thus, in 2005, there were no obstacles to a Prolog that extends

Schematic Prolog + CP(FT) + CP(R) … (1)

by including also CP(RT), CP(B), CP(Z), and CP(FD).

Yet the only forms of Prolog that existed at the time were basically the Prolog that emerged in the Seventies. The innovations beyond that point were restricted to implementation in WAM. The connection to constraint programming only existed by Prolog being a front-end to CP implementations. The software and logic architecture suggested by the equation (1) was still to come. So much for Senner’s suggestion of the development of Prolog as a relentless march forward. A decade went by before the great advances of the Nineties to logic programming were recognized as such and exploited accordingly. These were dark times indeed when XML, virtual machines, and constraint programming wandered around aimlessly, oblivious of their true destiny.

One can define a prime number as an integer greater than 1 that has no divisors other than 1 and itself. This is a declarative definition: it says what a prime number is rather than how to get one. One can also appeal to the Sieve of Eratosthenes. This would be a procedural definition. It is the opposite of declarative and it only tells how to get prime numbers.

In Software Engineering it is considered bad form to specify a function by pseudocode or by a reference implementation; a proper specification is supposed to be declarative. This blanket preference for the declarative assumes that such specifications are always better: easier to understand and to write. This is indeed the case with prime numbers, but not always. In the case of the Reef Knot, the Bowline, or any other kind of knot, a declarative definition, though perhaps possible, is likely to be less useful than being shown how to make one, which is procedural. Here I consider an example where it is easy to compare the merits of the Good (declarative), the Bad (procedural), and an even less respectable third approach.

Example: Scoring in “Mastermind”

Consider the game of Mastermind (trademark of Parker Bros.). In this game there are two players, the “Codemaker” and the “Codebreaker”, each on opposite sides of a board that is specific to this game. On the Codemaker side there are n slots in which code pegs are placed by the Codemaker. Each of these are in one of m colours (red, green, blue, …). The code pegs are concealed by a shield from the Codebreaker.

The Codebreaker tries to replicate the code on his side of the board. Each of these attempts takes the form of a probe, a sequence of n code pegs. On the Codebreaker’s side there is room for several of such probes. For each probe the Codemaker determines a score, which indicates how close the probe is to the code. The score consists of a set of “key pegs”, each of which is black or white. We are here concerned with specifying the scoring function, a function that maps a pair consisting of a code and a probe to the appropriate set of key pegs.

Suppose now that I want to write a program to compute the score from a given code and a given probe. Proper Software Engineering requires me to write a specification first so as to make it easier to write the program. Below you find a C program, which I find easy to write and to read. I have also made an attempt to write a declarative specification, which I find to be neither easy to write nor to read. This does not, of course prove that the Software Engineering method is wrong: I may just be singularly inept at writing specifications. All I can say is that the one shown below is the best I could come up with.

What is the basis of both of the program and of the specification? All we have to go on are the directions to prospective players given by Parker Bros., the manufacturer of the game:

Black key pegs are placed by the Codemaker in any of the key peg holes for every code peg placed by the Codebreaker which has the colour and is in exactly the same position as one of the code pegs behind the shield.

White key pegs are placed by the Codemaker in any of the key peg holes when any hidden code peg behind the shield matches the Codebreaker’s code pegs in colour only but not in position.

Consider the following situation:

     Codemaker     B  Y  G  B
     Codebreaker   Y  W  Y  Y

Here the Codebreaker has three yellow (Y) code pegs, all of which “match the Codemaker’s code pegs in colour only, but not in position”. Or is it only one?

The Parker Bros. are apparently aware of the difficulty, but have given up on trying to formulate an adequate description, because they add, rather lamely,

Example: If one red code peg is behind the shield and the Codebreaker places two red code pegs in the [sic] position, ONE white key peg is used.

The rule as formulated by Parker Bros has a declarative flavour. I will try harder than they have done in attempting to give an adequate declarative description. For comparison, I will also give procedural descriptions, in English as well as in the C programming language. All of these will be more complex and harder to read than the inadequate one by Parker Bros.

Finally, I will address the phenomenon that the customers of Parker Bros have no trouble in scoring. This is remarkable: Parker Bros does not tell them, so how do they know? Why do I think I know? Although it is not in the given description, we all seem to agree, and I bet Parker Bros also agrees.

That’s a mystery.

A procedural description in C

typedef enum{white, green, yellow, red, black, blue} colour;
typedef enum{false, true} bool;
typedef struct{int blacks; int whites;} score;

score scoreFn(colour code[], colour probe[], int n) {
  score s = {0, 0};
  bool cc[n];  // characteristic function for subset of code[]
  bool pp[n];  // characteristic function for subset of probe[]
  for(int i=0; i < n; i++) {
    if (code[i] == probe[i]) {
      s.blacks++;
      cc[i] = pp[i] = false;
      // this pair is counted; remove it
    }
    else cc[i] = pp[i] = true;
    // check this pair later for white scoring peg
  }
  for(int i=0; i < n; i++) {
    for(int j=0; j < n; j++) {
      if (cc[i] == true && pp[j] == true
          // both pegs present
          && code[i] == probe[j]) {
        s.whites++;
        cc[i] = pp[j] = false;
        // remove both pegs
      }
    }
  }
  return s;
}

A declarative specification

Given are sequences c and p of n colours. To specify scoring is to specify the functions b(c,p) and w(c,p) giving the numbers of black and white scoring pegs, respectively.

A strong match between c and p is a pair [c_i, p_i] such that c_i = p_i.

Two pairs [c_i, p_j] and [c_k, p_l] are disjoint iff i and k are not equal and j and l are not equal.

A weak match between c and p is a pair [c_i, p_j] such that it is disjoint from any strong pair and such that c_i = p_j.

b(c,p) is the number of strong matches between c and p.

Lemma: All maximal sets of mutually disjoint weak matches have the same size.

w(c,p) is the size of a maximal set of mutually disjoint weak matches.

An informal procedural description

If we don’t have hang-ups about the need for declarative specifications and if we don’t need to instruct a computer in the skill of scoring, then we can just say:

1. For each location where the code and probe have a peg of the same colour, remove both, and place a black scoring peg.
2. For each code pege and probe peg that have the same colour, remove both, and place a white scoring peg.

This works because it is procedural: it appeals to a state, and this state changes as a result of the prescribed actions. The action of removing pegs avoids the ambiguity in the directions given by the Parker Bros.

On the role of examples

Parker Bros relied on an example to supplement their attempt at a declarative specification. It is unlikely that this single example nails down all possible completions of the incomplete specification. Yet it seems to be enough for everyone to complete the specification and to arrive at the same completion. How is this possible? Is it perhaps so that we, as a species, have an innate sense of what is simplest, or most natural, and that this example was enough for all our brains to latch onto this? If this is so, then perhaps examples are all we need. It is the contention of Machine Learning that programs can be written on the basis of examples only, and automatically so. Maybe we can use examples instead of a definition rather than as a redundant supplement to an attempt at a self-sufficient definition.

It may be objected that examples can never be more than a hand-waving substitute for rules, justified only in situations where those whose job it is to emit the knowledge are too stupid to write rules or where those who need to absorb the knowledge are too stupid to understand them. It may be that examples can unambiguously determine the simplest function that is exemplified by the given examples.

I can imagine that we order all descriptions compatible with a given set of examples and find that the one with least Kolmogorov complexity is much simpler than the next simplest. Then we take this as the one that is defined by the examples. I can also imagine that our brains are so wired that we jump to the conclusion that this is the intended description.

Conclusions

For the concept of Prime Number, a declarative definition is easiest to give. In the case of a knot, a procedural one is easiest. The scoring function for the game of Master Mind is easier to define procedurally than declaratively, at least in my experience. In practice an incomplete declarative definition supplemented by an incomplete set of examples has been found satisfactory. This makes one wonder whether a function that is the least complex generalization of a set of examples would also work.

In 1966 I became a PhD student of A. van Wijngaarden at the Computational Department of the Mathematical Centre in Amsterdam. Whatever my topic was going to be, it was clear that my job would be to program. There used to be one computer, the Electrologica X1. Shortly before my arrival a much faster and bigger successor was added: the X8 from the same manufacturer. The language was Algol 60.

Most organizations have a strong attachment to a single programming language. At the Mathematical Centre this attachment was even stronger than usual. My new colleagues proudly pointed to the cover of the Revised Report on the algorithmic language Algol 60, which showed van Wijngaarden among the members of the committee that had defined the language. Moreover, they told me, as soon as the report was finalized early 1960, a race was on between Peter Naur and his team at Regnecentralen in Copenhagen and Dijkstra at the Mathematical Centre; upon which Dijkstra vowed not to shave until the project was finished; upon which Dijkstra finished the compiler by himself in six weeks, in assembler, on the fly inventing new compilation techniques (such as required for Call By Name), handily beating the Regnecentralen team.

Thus was the story that had gotten better in the telling ever since Dijkstra left the Mathematical Centre in 1962 to become a professor at THE, the engineering school in Eindhoven in the Netherlands. In 2000 I returned to what used to be the Mathematical Centre. During the year I was there, a visit by Dijkstra was announced. As I was curious about the origins of the Mathematical Centre’s early Algol 60 compiler, I e-mailed Dijkstra in advance the above story with the request to meet him to hear his side. I got a reply by e-mail, which is just as well because the brief return of the prodigal son to the Netherlands turned out to be a tightly orchestrated whirlwind media event.

Here is Dijkstra’s reply (my translation from the Dutch original):

Allow me to make a few corrections.

(i) I have never competed with Peter Naur. (I was raised somewhat puritanically: we had to try our hardest and give the most, but we were not allowed to compete. I am still grateful to my parents for this education.) When we had decided on the structure of the object program, the three of us (that is, Aad van Wijngaarden, Jaap Zonneveld and myself) went to Copenhagen to check our plans with Peter, just to make sure, before we started coding. Of course Peter’s group was the only one with which such a discussion was possible. I remember being surprised later (and somewhat disappointed) that Peter’s group only finished in the summer of 1961. By the way, Peter made this compiler, I think, not for the GIER [the machine built by Regnecentralen], but for a larger machine of Swedish origin. I thought that Peter’s compiler for the GIER was his second.

(ii) We were to implement with five of us: Aad van Wijngaarden, Jaap Zonneveld, Joke Feringa, Freek Barning, and myself. Feringa and Barning left quite soon. Van Wijngaarden did not contribute — on the contrary! — but pretended otherwise to the outside world. I had been warned by Bram Loopstra and Carel Scholten [hardware designers formerly at the Mathematical Centre] that the little chief was not adverse to showing off other peoples’ work as his own. I was not in the mood to let that happen to me. At that time Jaap, van W. and I were clean-shaven; Jaap had a moustache. To make clear that the two of us were doing the work, Jaap and I decided not to shave until the implementation worked, being convinced that van W. would not follow us in this. In this way we made it clear who was responsible for the Algol compiler. Thus it happened; that was around Pentecost [a European holiday six weeks after Easter]. In August the system worked. Jaap got rid of his moustache and beard soon after; I retained them.

(iii) The Algol compiler was, by the way, the occasion for me to bang my fist on the table in van Wijngaarden’s office. (I have not done so ever again.) Van W. wanted us to model the compiler on what Harry D. Huskey had shown us that fall. That was such a chaotic mess that I told van W. that if he wanted such a compiler, he would have to do it himself and that I refused to have anything to do with such a project. He backed down. (I think Niklaus Wirth in his Turing lecture made a remark about Huskey’s programming style; Huskey was his thesis adviser.)

With my cordial regards and my best wishes,
as ever yours Edsger

At least the myth that I picked up soon after my arrival at the Mathematical Centre, while mythical, turned out to be a fruitful one. In the fulness of time, it elicited an even more interesting story. I was pained to hear about van Wijngaarden’s role in this. As my thesis adviser he cared for me beyond the call of duty. He had been Dijkstra’s thesis adviser before that. In a handsome tribute to van Wijngaarden in his 1972 Turing lecture, Dijkstra acknowledges his debt.

I only remember having seen Dijkstra at the Mathematical Centre on one occasion during my stay there between 1966 and 1971. In 1968 he gave a lecture on the subject of the goto statement. It was summer and the lecture was scheduled in the evening; only half a dozen showed up. Next morning T.J. Dekker (known for the first implementation of semaphores) and I, puzzled, were trying to figure out how one could do my quicksort variant (ACM Algorithm 402) without goto’s.

In those years the most visible activity at the Computational Department of the Mathematical Centre was Algol 67, which segued into Algol 68. When the work did not make the implied deadline, they decided not to change the name another time. The success of Algol 60 made the official (sponsored by IFIP) successor a big deal. Dijkstra’s success with the Algol 60 compiler made him a key member of the committee appointed to define the successor. However, irreconcilable differences with van Wijngaarden caused Dijkstra to resign. I am told that the addition of semaphores to Algol 68 was a gesture to placate Dijkstra. Also, in 1971 when Dijkstra officially inaugurated the THE multiprogramming system, van Wijngaarden made a pilgrimage to Eindhoven, with as many of us in tow as he could muster. I think Dijkstra never realized how much he meant to the big “little chief”.

By 1973 I was in Edinburgh, learning logic programming from Kowalski. Enthused with the new religion, I planned a lecture tour through the Netherlands. I wrote to everyone I knew offering a talk on logic programming. Dijkstra said OK, provided it would be on Tuesday afternoon. To my surprise, the great man nearly fell off his chair nodding in agreement at several points in my talk. Afterwards he took me excitedly to his office to show his guarded-command formalism featuring the same type of nondeterminism he had noticed in the logic programs I had just presented.

In the train on my way to the next stop I tried to reproduce an example of program derivation that Dijkstra had shown to me.

It is required to ensure that m is the maximum of x and y. The natural formulation is

(m = x or m = y) and m >= x and m >= y

The crucial move is to apply distributivity to get

(m = x and m >= y) or (m = y and m >= x)

Then equality and commutativity to get

(x >= y and m = x) or (y >= x and m = y)

which transcribes to Prolog or guarded commands:

if x >= y -> m = x | y >= x -> m = y fi

On this, or a visit not long after, I ate with Edsger’s family seated at a big round dining table with a Lazy Susan in the middle loaded with the typically Dutch huge assortment of jars with stuff to smear or sprinkle on slices of bread, including jams, chocolate paste, and several varieties of “muisjes”, a cost- and time-effective way to still appetites of hungry teenagers. The backyard, flowerless, optimized for kids and dogs. Upstairs, the office. Edsger told me how pleased he was to be at the university only on Tuesdays, consulting with Burroughs for the rest of the time. He pointed proudly to the two telephones on his desk, one of which he could use to call anywhere in the world, with the bills going direct to Burroughs. To most people in the 1970s in the Netherlands this was a mind-boggling concept. It was hard enough to get one phone.

To the German general public, “der alte Fritz” means king Frederick the Great of Prussia. To the computer science community it meant prof. dr. F.L. Bauer, the pope of German computer science. Bauer was the professor in Munich and organized a long series of NATO Summer Schools. He loved the Bavarian country side and located the summer schools in Marktoberdorf, a small town away from Munich. These events were dominated by three stars: Dijkstra, Hoare, and Wirth; the rest of the lectures were filled in by protege’s of Bauer’s from Munich. It was all extremely European, something one couldn’t say of computer science as a whole.

In 1973, to get some variation, Alan Perlis was invited. He couldn’t be more different, and stole the show. The summer school was held in a boarding school. Breakfast consisted of excellent coffee (black in my case), exquisitely fresh crispy rolls, butter, and jam. For several of the American participants this caused considerable agony, one (who didn’t look particularly healthy) arguing that anything less than two ounces of protein for breakfast is detrimental to health. Lunch was also at the boarding school. For dinner we got vouchers at the local tavern. At opposite ends Dijkstra and Perlis were holding forth to the faithful, trying to ignore some of us shuttling back and forth.

Apart from the two telephones, another notable feature of the office in Dijkstra’s home was an elegant portable Olivetti typewriter. At the time, a proper professor would write with a fountain pen, possibly a Mont Blanc, which was at the time one of several ordinary brands. Only secretaries touched a keyboard. But Edsger typed his EWD memo’s himself, somewhere acknowledging the unversity for purchasing for his sole use the one typewriter with the one font that could do justice to his thoughts. This way he did a lot of typing: the big one, “Notes on Structured Programming” (EWD249), runs to 88 pages.

Fast forward to my visit to Edsger in the 1980s in Austin, Texas, when all profs did their writing on keyboards and Edsger refused to touch one. I have since been told that one way of swearing allegiance to Edsgerismus is to purchase a Mont Blanc and to find that it facilitates the proofs of your programs (which are only to be contemplated, not executed). In his office Edsger had a pencil dangling from a string with a sign pointing to it saying “Word Processor”.

Not only were word processors anathema to Edsger as superfluous mechanical devices, but they also gave offense by coming with
fat manuals.Edsger told me indignantly that some manual for Wordperfect ran tosix hundred and fifty pages. He reminded me of the fact that Georg Joos only needed six hundred for his great work “Handbook of Theoretical Physics”. It was blasphemy to Edsger that a manual for buggy software for a perverse purpose used more space than a comprehensive compilation of the most sublime knowledge. If you already have a Mont Blanc, consider for your next purchase this book of Joos’s. Of course, it’s long been out of print. But it can be ordered from several antiquarian bookstores via the Web. For true Edsgerismus, get it as “Handbuch der Theoretischen Physik”.

At the time of his 2000 visit to the Netherlands, Edsger was already being treated for what turned out to be his fatal illness. The next year he moved to his old home near Eindhoven where I had first visited him in 1973. He died there in 2002.