An Interview with Paul McJones

In my quest for pioneer programmers I was fortunate to get in touch with one who has a possibly unrivalled record of participation in projects that are important in the development of computing over the decades. I am speaking of Paul McJones, who was part of the team assembled by Butler Lampson to develop Cal TSS, one of the first operating system designs to tackle the problems of protected and even mutually-suspicious subsystems. At IBM San Jose Research he worked with John Backus on his RED languages and with the System R group on the first relational database system. At Xerox he participated in the development of the Star office automation system (based on PARC’s Alto personal computer). Via Tandem Computer he went to do research at DEC SRC in Palo Alto. After various Silicon Valley start-ups he went to work at Adobe. This included a collaboration with Alexander Stepanov. A spin-off of this research is Elements of Programming (by A. Stepanov and P. McJones, Addison-Wesley, 2009), which presents a novel methodology that spans the gap between abstract mathematics and efficient algorithms.

In the following interview we try to cover at least a little bit of this wide and varied terrain.

APP
I first heard of you as part of the team of the CAL time-sharing system in Berkeley. That must have been around 1969. Is that right? Were you still a student at that time?

McJones
Yes, I came to Berkeley in the fall of 1967 as a freshman.

APP
Did you get started on programming in connection with a course? Which?

McJones
No, it wasn’t in a university course. At high school a dedicated physics teacher taught me programming.

APP
Ah! I’m sure that’s a great way to get started. Tell me more.

McJones
Actually, my first taste of programming was even earlier, again because of a dedicated teacher. In the seventh grade, a teacher (now Dr. Marilyn Fendrick; then Mrs. Benefield) encouraged me to pursue my curiosity about digital logic and computers. I read a book called Basic Computer Programming by Theodore G. Scott. This exposed me to the basic ideas of computers, but without a real machine to program, I explored analog and digital electronics for the next few years.

In high school, I took a course called Analytical Techniques in the Physical Sciences, designed and taught by physics teacher Carl Duzen. It included some linear algebra, solving differential equations on an analog computer, and programming a digital computer in FORTRAN IV and assembly language. We used keypunches at a company nearby and an IBM 7094 at an aerospace company in Los Angeles. Duzen only had time to drive to the aerospace company about once a week, so the turn-around time was very long!

Later that year, Jack Perrine, an independent software consultant, visited Duzen’s class and hired me and another student as after-school interns. Perrine had been a member of the team at Computer Sciences Corporation that wrote the original UNIVAC 1107 FORTRAN compiler. Eventually Perrine tired of corporate life, and founded ATHENA Programming to support the compiler. My assignments were to fix bugs in the runtime and to write utility routines such as snapshot dumps — this was my real introduction to assembly language programming. So by the time I entered Berkeley that fall, my experience at ATHENA helped me get an entry-level programming job at the campus Computer Center.

APP
What kind of work did you do there?

McJones
At first it was similar to what I’d done at ATHENA: writing a library subroutine to read very long magnetic tape records, and a utility program to catalog the contents of a tape. The spring of my freshman year, I got interested in Snobol3. It so happened that Charles Simonyi had entered Berkeley the same time as I did. He already had a great deal of experience writing compilers in Hungary and at Regnecentralen in Denmark. At Berkeley he proposed to the Computer Center that he write a from-scratch implementation of Bell Lab’s brand-new Snobol4. The Bell implementation used a virtual machine implemented in macros that was very portable and elegant, but too inefficient in memory and cycles for running student jobs. Charles’s plan was to write in assembly language (for the CDC 6400), base the compiler on the GIER design, and keep the entire system “lean and mean”. Someone suggested I work with Charles; he agreed, and I jumped at the opportunity.

APP
So you were off and running, working with one of the greats in programming, and that in the first year of University! Did you have any time to take courses?

McJones
I was enrolled full-time in the electrical engineering program, enjoying the social life in a dormitory with new friends with diverse backgrounds, riding my motorcyle around the Bay Area, and spending lots of time programming. To be honest, my grades suffered the first year. Late in the year, I met my future wife (at the Xerox machine, while copying the Snobol3 manual), and her good influence helped my grades recover.

APP
What was the fate of the Snobol system? Such projects often get lost in management muddles.

McJones
We built a successful system — if I remember correctly, it was limping along that fall — that was used at Berkeley and other CDC installations well into the 1970s because of its efficiency and despite its lack of full compatibility. Charles created a sound architecture and wrote a prodigious amount of code. (It just occurred to me that he wrote a prototype of the pattern matcher in ALGOL 60). I wrote I/O routines that took advantage of the asynchronous I/O capability of the SCOPE operating system APIs, so a one-line copy program written in our Snobol would keep tapes running at full speed, unlike some of the standard utilities. While our implementation omitted a number of standard Snobol4 features (e.g., the TABLE data type), it also added some new ideas. For example, we wanted to be able to write a symbolic debugger in Snobol4, as one could do in LISP. This required being able to decompose every compound data value, such as the elaborate Snobol4 patterns. So we added appropriate built-in predicates and selectors.

Snobol4 was used in computer science courses since it allowed creating data structures — a role later subsumed by Pascal. It was also used in a course for humanities students since it had good tools for processing text; the lecture notes included an appendix serving as the language reference manual.

A couple of years ago, through my interest in the history of software, I was able to look through the electronic archives of the late Ralph Griswold, one of the Snobol designers. I was amused and pleased to find a copy of the source code to our Snobol system, and even more amused when I was able to assemble and run it on an emulator for the CDC 6600.

APP
When you tell of keypunch days with long turnarounds, I wondered what else you did in addition to working over listings. Text with paper and pencil, of course. Did you use diagrams? Blackboards?

McJones
Schooled as I was in Fortran and assembly code, Algol control structures were a revelation. I wrote Algol-like pseudo code for my initial tasks at the Berkeley Computer Center and inserted it as comments interspersed with the actual assembly language. We drew lots of data structure diagrams with boxes and arrows. Somewhere I still have a photocopy of the ones Charles did for our Snobol’s runtime system: strings, numbers, patterns, and so on. We probably used blackboards some, but I think we often just wrote on the back of an old listing. Here‘s an example of a finished design from those days, complete with lots of diagrams.

APP
After interactive, paper-based teletypes and such the video terminal was an important transition for coders. Was there composition on the keyboard right away? Or was it still via pencil and paper first?

McJones
I first used video terminals at IBM, around 1975. They were in a common terminal room, so I’m pretty sure I often composed ahead of time via pencil and paper. I did have an IBM 2741 Selectric-style terminal in my office, but I think I tended to walk down to the terminal room most of the time — it was a useful way to “share state” with other team members. In late 1976 I went to Xerox. We used Altos for program development from the beginning. There was a brief attempt to have us share them, but we quickly achieved one Alto per person. I still wrote out my programs on paper; among other things, that meant I could work on them in the evening, many miles away from Xerox.

APP
Was the transition to IDE’s significant?

McJones
Most contemporary programmers love them, but I think IDEs always have a struggle dealing with the size and complexity of the largest systems. For example, it may still take 30 minutes to rebuild a complex operating system, even after making a small change.

APP
Because the tools of work have changed so much, one wonders how that influenced productivity, quality, or even thought itself?

McJones
In many programming shops today there is such a focus on speed: very frequent releases; programmers are measured by how much code they write or checkins they perform — do people have time to think? Software developers have become very good at testing, but notions of specification and correctness at design time often seem to be ignored.

APP
Speaking of tools, how about languages. In the first decades, there seem to have been two great transitions: in numerical work from assembler to FORTRAN, and in systems programming from assembler to C. How do these transitions compare?

McJones
They are hardly comparable. The transition to FORTRAN for numerics was extremely fast. Here‘s a September 1957 memo from Backus to his management with statistics showing a very impressive adoption rate only about six months after FORTRAN I shipped. FORTRAN allowed scientists and engineers to work at a fairly natural level of abstraction (numbers and arrays) without paying a performance penalty because of excellent code generation. FORTRAN quickly spread to most computers (rarely with the same quality of code generation), giving rise to the notion of portability. The first edition of McCracken’s book on FORTRAN programming came out in 1961; college courses taught the language to engineers and scientists. All this created a pretty strong positive-feedback loop.

APP
And why is the transition from assembler to C in systems programming hardly comparable?

McJones
It took much longer — roughly two decades — and involved many intermediate languages. I’m not sure quite why, but I’ll speculate that some of the sources of the delay were:

1. Evolution of machine architectures toward the eventual standard byte-addressed memory, uniform integer and floating data formats, and banks of general-purpose registers (the C machine!).
2. Evolution of language features allowing expressive, efficient programming of algorithms (e.g., iteration and selection statements), data structures (record/structure definitions; pointer types and pointer arithmetic, etc.), and management of complexity (preprocessors, modules, abstract data types, polymorphism of various kinds, etc.).
3. Realization by professional programmers (compared with scientists and engineers who program) of the advantages of a standard, portable programming language.

Along the way, people tried out ALGOL 58 dialects (NELIAC, MAD, and JOVIAL), ALGOL 60 extensions (Burroughs ESPOL, SAIL, etc.), ALGOL 60 follow-ons (ALGOL W, Pascal, Mesa, Modula), PL/I, Bliss, BCPL, and even FORTRAN. By the early 1980s, C was emerging as the standard: it was a good match for the standard machine architecture, and expressive enough for disciplined programmers to use for a wide variety of applications and systems. It spread along with Unix, and was relatively simple to port or reimplement on non-Unix operating systems including MS-DOS and Macintosh OS. Since then, C++ has joined C both by being a “better C” and by providing powerful tools for user-defined types.

APP
How about the transition to Object-Oriented Programming?

McJones
While Simula-67 introduced classes, inheritance and virtual procedures, Smalltalk popularized them. In the 1980s, many programmers wanted to try object-oriented programming and many language designers obliged them. Pundits convinced management this would solve all their problems. Java and object-oriented scripting languages came along in the 1990s with garbage collection and huge libraries: for a time, programming appeared to be a simple matter of copy-and-paste. But I think the pendulum is swinging the other way: people once again realize they need to focus on algorithms and data structures as well as “glue”. Inheritance is a programming technique, good for some things but not others.