A Curmudgery On Programming Language Trends

The current problems of software development and maintenance are primarily those of scale--hundreds of programmers scattered over tens of years tending to millions of lines of code. Problems like these aren't addressed by trendy tinkering with programming languages.

There's a programming folk theorem that what feels good to one programmer writing a 10,000-line program in isolation will be good for 100 programmers writing a one-million-line program as a team. Experience has shown this is false. In fact, features like self-modifying code, signature coercions, and programmer-defined operator overloading make a language feel good to the lone programmer but get in the way of effective team programming.

Historically we've focused on the productivity of individual programmers and developed tools to make this person more productive. We reasoned that if each member of a team were more productive, then the team would be more productive, too. But the product of a team is supposed to be something more than the sum of its members' products. If technology increases the productivity of individual team members but frustrates the crystallization of the team, it cannot address the real problems of software engineering.

In short, a chemical plant isn't simply a scaled-up copy of the chemist's laboratory; a one-million-line program isn't a concatenation of 10,000-line programs; and a programming team isn't just a midnight hacker with a lot of fingers.

Object-oriented programming will soon join the modular programming of the '60s, portable programming of the '70s, and structured programming of the '80s on the paradigm compost heap. One after the other, these breakthroughs to common sense have been converted into reasons to engage in massive translation of perfectly functional code.

These pitches always have a convincing ring. "It slices, it dices, and it does it all with modular components so you can add extensions later." But before we get extensions, a brand-new tool is introduced, and herein lies the real rub. Sooner or later, the folks who pay the bills are going to ask if buying the new code-o-matic is really a major advance over the knife; if translating the software problem from one language to another contributes to solving the problem, or has itself become part of the problem. While we've told the bill payers about information hiding and how we can change the implementation without changing the interface, it always seems we have to rewrite the whole thing. "This will be the last time...honest" is one of programming's big lies.

We've yet to compute whether any of these rewriting frenzies have paid for themselves because we're afraid the people paying the bills won't like the answer. If you take into account programmer time and capital resources (computers, disk space, and the like) and ignore opportunity loss (programs and features that could have been written and sold or used in the "old" programming language), I seriously doubt that these programming fads have been cost effective. What they have done is enrich and entrench the programming priesthood.

Can you imagine chemists changing the symbols of the elements or the diagrams of organic molecules as often as we change programming languages? Can you imagine mathematicians changing their notation every ten years and the confusion it would cause? Does this constant change of notation and methodology account for some of software's chaos? Does := == = = <- or not?

Programmers perpetually pretend (at least in front of their bosses) that the current programming fad is the last, and that although they were wrong the last five times, this system rewrite is really the last one. They rarely ask the question, "How hard is it going to be to free my programs from this language fad and move them to the next one or back to the previous one?"

Object-oriented methodology has made a number of positive contributions to how we think about programming and programming languages. However, the de facto object-oriented programming language winner, C++, is not one of them. How hard is it going to be to free our programs from C++? I believe that C++ will leave behind a bigger software mess than PL/I, Ada, and Lisp combined, for two reasons: 1. We lack a complete understanding of the discipline needed to use object-oriented programming effectively; and 2. C++ has a feature-creep history, nasty syntax, and complex semantics. And when we decide to escape from C++, the following land mines await us.

Programming with puns. Overloaded operators, multiple inheritance, and coerced function calls will be big problems for the C++-to-whatever team. The meaning of any symbol and where control goes when it is encountered depends in complex ways on the entire corpus of code present during compilation and linking. These language-extension and polymorphic-pun features of C++ make reading and debugging C++ difficult, particularly when the code has been written by a team of programmers. They may make translation--particularly automatic translation--impossible to any practical extent.

while(C) {version = C++}. C++ is turning out to be a royal dynasty of programming languages rather than just one well-defined programming language. Its syntax and semantics have changed repeatedly over the ten-plus years of its adolescence. To free a program from C++, you'll have to know which version of C++ and maybe even which C++ compiler it was written for. This was true of "classic" languages too, but the differences were apparent in the code syntax itself. With C++, the differences are often hidden in the semantics--how the same symbols are interpreted differently by the compilers and the run-time engine. (By the way, I hear there's a new version due out real soon now with templates, exceptions, and threads. Do you mean the American or European version?)

Anything you can do I can do IsA. Object-oriented programming methodology is an open invitation to gratuitous generalization and the growing of formal gardens of complexity kudzu. Indeed some OO wizards recommend that you add code and levels of generality to object-oriented programs simply on the grounds of completeness and possible future need. But the thrust of commercial programming is to do more with less, not less with more. The idea is to synthesize, to make programs smaller, to refine and focus them. If what you really want is an omelette, you don't start with a Faberge egg.

Admiral Grace Hopper, America's first and finest software engineer, said that we really don't do anything new in computing, we just give new names to old ideas. Consider the following:

  Hollerith    Date     Stroustrup   --------------------------------

  Field        Column   Attribute   Drum Card    Table    Class   Card         Row      Object
  Gang Punch   Join     Inherit

What is this concept that we rediscover with such regularity? In its purest form it is the familiar entity-attribute-relation (E-A-R) diagram. OOP is simply the realization of this idea in a programming language; the relational data model realizes it in a data store; and TAB cards and tabulating machines realize it in hardware.

In drawing up an E-A-R diagram for a program (doing object-oriented design, if you must be seen at a table at Maxim's), you're led inexorably toward a study of the problem and its context and away from the solution and its technology. This--and not multiply inherited, polymorphic, virtual mix-in friends--is the key insight of object-oriented methodology: Use E-A-R diagrams to do program design. After all, the understanding of and insight into the problem is the wellspring of program simplicity and implementation efficiency. A good architect studies bricks once, but studies people every time he designs a new building.

Programming languages have not proven to be particularly good tools for understanding problems. Or, said another way, the better a programming language is at problem description and study, the worse it is as a programming language. Furthermore, using the same notation to record your understanding of a problem and the program that's going to harness that understanding may not be a good idea. (There's probably a reason a printed circuit board or a chip mask doesn't look like a schematic diagram.)

Call it something new (like an "object") if you must, but understand that the idea has been around for a long time. Don't be afraid to learn about the idea by studying it under its older names. And while you're dipping into programming history, keep in mind that the whole point of the Jacquard loom was software reuse.

So how do programming at scale and E-A-R diagrams come together? How do you build large programs with E-A-R diagrams?

Object-oriented programming's use of E-A-R diagrams provides us with only one constructor--the "is a" (IsA) relation. Unfortunately, this really isn't the kind of constructor we need to build big programs. It's flat--it never gets off the ground. A generic car is about the same size as a 1957 Chevrolet, and a motorized transportation device is probably about the same size as a generic car. IsA lets us build vaulting conceptual castles, but it doesn't let us build parts to build parts with which to build real castles. And we need fleets, traffic jams, and parking lots of cars and cities with car fleets, traffic jams, and parking lots.

To do object-oriented programming at scale, we need more relations (the "R" in E-A-R), particularly the "part of" (PartOf) relation. Furthermore, to make PartOf work right, connecting a bunch of objects should lead to another object. In E-A-R talk, such a composite entity is called a "container." It is still a first-class entity which can participate in still further relations, including more PartOf relations. In its Hopper-esque rush to be new, OOP hasn't stopped to learn from its roots.

E-A-R languages also have to externalize; that is, they must give back to the programmer the relation-following machinery. In fact, I'd argue that the "methods" used to follow relations between objects are more important for building big programs than the methods used to compute some feature within an object. To convert an E-A-R diagram to a flowchart, just replace each box with a line and each line with a box. Code is nothing more than the way we maintain relations between data.

My father was a weekend handyman. Every Saturday morning started with a trip to the hardware store to find exactly the right tools to do the weekend's project. Thirty-degree ratchet wrench, underwater silicon glue, ultrasonic varnish remover. Every task had to have just the right tool to accomplish it. On Monday morning, Mom called the local professional handyman, who came with the same set of well-worn tools that he took to every job to clean up and finish up Dad's mess.

Programmers by disposition are enamored of technology. This is a strength when it comes to abandoning old, inefficient ways and adopting new, more efficient ways. But it's a weakness when new software tools arrive faster than we can learn to use them effectively and we spend too much time at the wrong end of their learning curves. The benefits of a truly effective new technology are foregone when, in the rush to try out the next shiny whiz-bang tool, we abandon the current one.

Most new programming-language technology is either yesterday's forgotten failures or today's common sense. Here's a sampling of the latter:

• Seek constructs that help with problems of scale; features that just gild the putty waste your time and usually make matters worse.
• Study the problem and understand its context; it is here and not in your high-tech toolkit that elegant solutions will be found.
• Assemble programs and systems from big parts; try to write as little code as possible; work the code, not the debugger.
• Pick your programming tools carefully, change them infrequently, and make sure you get to the top of the learning curve.

Al Stevens

Editor's Note: Scott's curmudgery on language trends caused DDJ contributing editor Al Stevens to look back over his years at the keyboard. Here's his reaction to Scott's article; we'd like to hear yours.

Scott Guthery's article, "A Curmudgery on Programming Language Trends" stirred feelings of deja vu. Scott's sentiments recall similar beatings that occurred in my breast with some regularity over the years. There have been a lot of those years. Scott and I started at about the same time in the late '50s: the Eisenhower years, cars with fins, duck tails, white bucks, Thunder Road, Bo Diddley, and Bird. His words move me to consider why those of our generation so rigorously resist change. I've lived and suffered through almost every paradigm shift known to the commercial data-processing craft. I rebelled at almost every one. I was almost always wrong.

My first job was as a "card walloper," an operator in an EAM shop. EAM means "electronic accounting machine," and it refers to a class of punched-card processing machines. We programmed them by wiring plug boards, and we operated them by hand-feeding cards into their hoppers and removing those cards from their stackers. We were in the data flow. I liked that work. We designed, programmed, and operated systems, often completing all three steps in the course of one duty shift.

When they made me a programmer on the IBM 650 and 1401, I accepted because it was an honor to be selected and it paid more, but I didn't like the idea. Computer programmers wore ties, sat at desks, worked normal hours, and operated the machines only when they were debugging programs. I didn't want to be so far away from the hardware. Once the programmer's job was done, someone else got to play with the gear. I learned machine code and then the SOAP II and Autocoder assembly languages. I didn't like assembly language. It took away the fun because it calculated the operation codes and operand addresses, something we had been doing for ourselves. Assembly language was supposed to make us more productive, though.

Soon we had a 1410, which had a complex input-output architecture. IBM brought a macro system named IOCS (Input-Output Control System) into our Autocoder shop. It would make programming easier, they said, make us more productive. The macros took care of reading and writing card, disk, printer, and tape records and testing for errors. This is no good, we said. They're taking the fun away again. We're supposed to forget about how the electromechanical devices work and concentrate more on the program's purpose.

Next came Cobol. It would not only make us more productive, it would increase the number of computers that we could program. I didn't like it. I thought it was awful, not for the reasons that programmers don't like Cobol now, but because it completely divorced the programmer from the hardware. How could you write an efficient program if you didn't manipulate the registers, if you didn't even know what they were?

Along with Cobol came the Closed Shop paradigm. It was supposed to make the computer, not us, more productive. Programmers were no longer allowed to see the machine, much less operate it. We passed decks of cards through a window--a real hole in the wall, not a Window--and got source, output, and memory-dump listings back through a window. These were the dark days of programming. This was one of the few rebellions that was in the right. It drastically reduced programmer productivity because we dealt with one bug per throughput, usually measured in days, all of which was okay to the empty suits because it gave the illusion of optimized computer usage, a more costly and valued commodity than programmer hours.

The Closed Shop paradigm was in tandem with the Big System paradigm. Management was enamored with the flawed idea that a computer software system should support every functional and performance requirement they could dream up, and that its first version should do it all. No requirement was negotiable, and none could be postponed for a later version. We saw, and still see, mainframe installations that cannot run uninterrupted for more than a few hours or days without the attention of a staff of systems programmers, database administrators, and applications programmers.

Minicomputers and, later, microcomputers both returned us to the old ways. They had primitive operating systems and assemblers and, at first, no so-called "high-level" languages. Programmers from my generation loved them. We had been spoiled--had gotten lazy and fat--by the compilers, database managers, and automatic flowcharters, and we fled back to our Spartan roots in droves. But ontogeny recapitulates philology, and both times the return to the golden years was short-lived.

Both structured programming and the relational-database model made sense right out of the chute because they directly addressed and partially solved common and persistent design and development problems, and their superiority was obvious. They were valid, and they worked. They have problems of their own, but it's worth it to benefit from what they offer.

I looked at object-oriented programming in the middle '80s. I saw nothing compelling. The problems I dealt with did not cry for a change in methods. I wrote programs in C and the programs worked. Why change? But when it was inevitable that the C++ star would rise, another look was called for. I was skeptical. Without experience in other object-oriented languages, I found in C++ a better method for expressing programs. Part of that is due to the notational improvements that C++ brings to C. Another part is the ability to build new data types that behave as if they belong to the language. Still another part is the lure to regard and express software solutions from the perspective of the data types rather than the functions.

Like C, C++ offers the programmer more freedom than discipline. But discipline in programming is a personal matter, and without it a programmer in any language is going to build bad programs, and a team of programmers is going to build bad software systems. Which brings us to the real problem: C++ and object-oriented programming do not correct the Big System paradigm. If you must build a big system, then no language, no tool, no paradigm is going to guarantee success. The size of the system, the complexity of the relationships between data entities, and the number of people doing the design and construction are what get in the way.

For example, I spent 1974 to 1978 in an office building in Roslyn, Virginia helping to build a network of ten minicomputers with 20 microcomputer workstations to process text-message traffic. Dozens of people and several contractors and government agencies were involved. In those four years, the project saw slipped schedules, blown budgets, and changes in direction. During that same time, I observed from my window the ground breaking, construction, and completion of a Hyatt hotel, several new office high-rises, and the new underground Metro mass-transit system. When I left, the hotel, office buildings, and subway were done and operating, but the computer system was still foundering.

Despite all the attempts and all the new and improved methodologies, we have never figured out how to manage big software projects to a successful, planned, and orderly completion. The failure of a new paradigm to solve what has never been solved before does not by itself invalidate the paradigm.

Scott observes that chemists and mathematicians do not change their notations as frequently we do. Neither do construction engineers, doctors, astronomers, geologists, biologists, and archeologists. Consider the age of their sciences and that of our craft. Their notations are many hundreds of years old. Ours is less than 50. When Scott and I started programming, the craft was less than ten years old. There were as many programmers in the world than as attend a typical small trade show today. The computing power that then filled a room now fits in your pocket. The rapid changes in technology and the sheer number of people involved demand and assure that the tools and techniques will change and improve. The programming industry will grow. We cannot afford to stop growing with it.

Copyright © 1992, Dr. Dobb's Journal