A Brief History of Craftsmanship in Programming
This chapter is from the book
This chapter is from the book
This chapter is from the book
)
The dream of flying is almost certainly as old as humanity. The ancient Greek myth describing the flight of Daedalus and Icarus dates from around 1550 BCE. In the millennia that followed, a number of brave, if foolish, souls have strapped ungainly contraptions to their bodies and leapt off cliffs and towers to their doom in pursuit of that dream.
Things began to change about five hundred years ago when Leonardo DaVinci drew sketches of machines that, though not truly capable of flight, showed some reasoned thought. It was DaVinci who realized that flight could be possible because air resistance works in both directions. The resistance caused by pushing down on the air creates lift of the same amount. This is the mechanism by which all modern airplanes fly.
DaVinci’s ideas were lost until the middle of the eighteenth century. But then began an almost frantic exploration into the possibility of flight. The eighteenth and nineteenth centuries were a time of intense aeronautical research and experimentation. Unpowered prototypes were built, tried, discarded, and improved. The science of aeronautics began to take shape. The forces of lift, drag, thrust, and gravity were identified and understood. Some brave souls made the attempt.
And some crashed and died.
In the closing years of the eighteenth century, and for the half century that followed, Sir George Cayley, the father of modern aerodynamics, built experimental rigs, prototypes, and full-sized models culminating in the first manned flight of a glider.
And some still crashed and died.
Then came the age of steam and the possibility of powered manned flight. Dozens of prototypes and experiments were performed. Scientists and enthusiasts alike joined the gaggle of people exploring the potential of flight. In 1890, Clément Ader flew a twin-engine steam-powered machine for 50 meters.
And some still crashed and died.
But the internal combustion engine was the real game-changer. In all likelihood, the first powered and controlled manned flight took place in 1901 by Gustave Whitehead. But it was the Wright Brothers who, on December 17, 1903, at Kill Devil Hills, North Carolina, conducted the first truly sustained, powered, and controlled manned flight of a heavier-than-air machine.
And some still crashed and died.
But the world changed overnight. Eleven years later, in 1914, biplanes were dogfighting in the air over Europe.
And though many crashed and died under enemy fire, a similar number crashed and died just learning to fly. The principles of flight might have been mastered, but the technique of flight was barely understood.
Another two decades, and the truly terrible fighters and bombers of World War II were wreaking havoc over France and Germany. They flew at extreme altitudes. They bristled with guns. They carried devastating destructive power.
During the war, 65,000 American aircraft were lost. But only 23,000 of those were lost in combat. The pilots flew and died in battle. But more often they flew and died when no one was shooting. We still didn’t know how to fly.
Another decade saw jet-powered craft, the breaking of the sound barrier, and the explosion of commercial airlines and civilian air travel. It was the beginning of the jet age, when people of means (the so-called jet set) could leap from city to city and country to country in a matter of hours.
And the jet airliners tore themselves to shreds and fell out of the sky in terrifying numbers. There was so much we still didn’t understand about making and flying aircraft.
That brings us to the 1950s. Boeing 707s would be flying passengers from here to there across the world by the end of the decade. Two more decades would see the first wide-body jumbo jet, the 747.
Aeronautics and air travel settled down to become the safest and most efficient means of travel in the history of the world. It took a long time, and cost many lives, but we had finally learned how to safely build and fly airplanes.1
Chesley Sullenberger was born in 1951 in Denison, Texas. He was a child of the jet age. He learned to fly at age sixteen and eventually flew F4 Phantoms for the Air Force. He became a pilot for US Airways in 1980.
On January 15, 2009, just after departure from LaGuardia, his Airbus A320 carrying 155 souls struck a flock of geese and lost both jet engines. Captain Sullenberger, relying on over twenty thousand hours of experience in the air, guided his disabled craft to a “water landing” in the Hudson River and, through sheer indomitable skill, saved every one of those 155 souls. Captain Sullenberger excelled at his craft. Captain Sullenberger was a craftsman.
The dream of fast, reliable computation and data management is almost certainly as old as humanity. Counting on fingers, sticks, and beads dates back thousands of years. People were building and using abacuses over four thousand years ago. Mechanical devices were used to predict the movement of stars and planets some two thousand years ago. Slide rules were invented about four hundred years ago.
In the early nineteenth century, Charles Babbage started building crank-powered calculating machines. These were true digital computers with memory and arithmetic processing. But they were difficult to build with the metalworking technology of the day, and though he built a few prototypes, they were not a commercial success.
In the mid-1800s, Babbage attempted to build a much more powerful machine. It would have been steam powered and capable of executing true programs. He dubbed it the Analytical Engine.
Lord Byron’s daughter, Ada—the Countess of Lovelace—translated the notes of a lecture given by Babbage and discerned a fact that had apparently not occurred to anyone else at the time: the numbers in a computer need not represent numbers at all but can represent things in the real world. For that insight, she is often called the world’s first true programmer.
Problems of precise metalworking continued to frustrate Babbage, and in the end, his project failed. No further progress was made on digital computers throughout the rest of the nineteenth and early twentieth centuries. During that time, however, mechanical analog computers reached their heyday.
In 1936, Alan Turing showed that there is no general way to prove that a given Diophantine2 equation has solutions. He constructed this proof by imagining a simple, if infinite, digital computer and then proving that there were numbers that this computer could not calculate. As a consequence of this proof, he invented finite state machines, machine language, symbolic language, macros, and primitive subroutines. He invented, what we would call today, software.
At almost exactly the same time, Alonzo Church constructed a completely different proof for the same problem and consequently developed the lambda calculus—the core concept of functional programming.
In 1941, Konrad Zuse built the first electromechanical programmable digital computer, the Z3. It consisted of more than 2,000 relays and operated at a clock rate of 5 to 10Hz. The machine used binary arithmetic organized into 22-bit words.
During World War II, Turing was recruited to help the “boffins” at Bletchley Park decrypt the German Enigma codes. The Enigma machine was a simple digital computer that randomized the characters of textual messages that were typically broadcast using radio telegraphs. Turing aided in the construction of an electromechanical digital search engine to find the keys to those codes.
After the war, Turing was instrumental in building and programming one of the world’s first electronic vacuum tube computers—the Automatic Computing Engine, or ACE. The original prototype used 1,000 vacuum tubes and manipulated binary numbers at a speed of a million bits per second.
In 1947, after writing some programs for this machine and researching its capabilities, Turing gave a lecture in which he made these prescient statements:
We shall need a great number of mathematicians of ability [to put the problems] into a form for computation.
One of our difficulties will be the maintenance of an appropriate discipline, so that we do not lose track of what we are doing.
And the world changed overnight.
Within a few years, core memory had been developed. The possibility of having hundreds of thousands, if not millions, of bits of memory accessible within microseconds became a reality. At the same time, mass production of vacuum tubes made computers cheaper and more reliable. Limited mass production was becoming a reality. By 1960, IBM had sold 140 model 70x computers. These were huge vacuum tube machines worth millions of dollars.
Turing had programmed his machine in binary, but everyone understood that was impractical. In 1949, Grace Hopper had coined the word compiler and by 1952 had created the first one: A-0. In late 1953, John Bachus submitted the first FORTRAN specification. ALGOL and LISP followed by 1958.
The first working transistor was created by John Bardeen, Walter Brattain, and William Shockley in 1947. They made their way into computers in 1953. Replacing vacuum tubes with transistors changed the game entirely. Computers became smaller, faster, cheaper, and much more reliable.
By 1965, IBM had produced 10,000 model 1401 computers. They rented for $2,500 per month. This was well within the reach of midsized businesses. Those businesses needed programmers, and so the demand for programmers began to accelerate.
Who was programming all these machines? There were no university courses. Nobody went to school to learn to program in 1965. These programmers were drawn from business. They were mature folks who had worked in their businesses for some time. They were in their 30s, 40s, and 50s.
By 1966, IBM was producing 1,000 model 360 computers every month. Businesses could not get enough. These machines had memory sizes that reached 64kB and more. They could execute hundreds of thousands of instructions per second.
That same year, working on a Univac 1107 at the Norwegian Computer Center, Ole-Johan Dahl and Kristen Nygard invented Simula 67, an extension of ALGOL. It was the first object-oriented language.
Alan Turing’s lecture was only two decades in the past!
Two years later, in March 1968, Edsger Dijkstra wrote his famous letter to the Communications of the ACM (CACM). The editor gave that letter the title “Go To Statement Considered Harmful.”3 Structured programming was born.
In 1972, at Bell Labs in New Jersey, Ken Thompson and Dennis Ritchie were between projects. They begged time on a PDP 7 from a different project team and invented UNIX and C.
Now the pace picked up to near breakneck speeds. I’m going to give you a few key dates. For each one, ask yourself how many computers are in the world? How many programmers are in the world? And where did those programmers come from?
1970—Digital Equipment Corporation had produced 50,000 PDP-8 computers since 1965.
1970—Winston Royce wrote the “waterfall” paper, “Managing the Development of Large Software Systems.”
1971—Intel released the 4004 single-chip microcomputer.
1974—Intel released the 8080 single-chip microcomputer.
1977—Apple released the Apple II.
1979—Motorola released the 68000, a 16-bit single-chip microcomputer.
1980—Bjarne Stroustrup invented C with Classes (a preprocessor that makes C look like Simula).
1980—Alan Kay invented Smalltalk.
1981—IBM released the IBM PC.
1983—Apple released the 128K Macintosh.
1983—Stroustrup renamed C with Classes to C++.
1985—The US Department of Defense adopted waterfall as the official software process (DOD-STD-2167A).
1986—Stroustrup published The C++ Programming Language (Addison-Wesley).
1991—Grady Booch published Object-Oriented Design with Applications (Benjamin/Cummings).
1991—James Gosling invented Java (called Oak at the time).
1991—Guido Van Rossum released Python.
1995—Design Patterns: Elements of Reusable Object-Oriented Software (Addison-Wesley) was written by Erich Gamma, Richard Helm, John Vlissides, and Ralph Johnson.
1995—Yukihiro Matsumoto released Ruby.
1995—Brendan Eich created JavaScript.
1996—Sun Microsystems released Java.
1999—Microsoft invented C#/.NET (then called Cool).
2000—Y2K! The Millennium Bug.
2001—The Agile Manifesto was written.
Between 1970 and 2000, the clock rates of computers increased by three orders of magnitude. Density increased by four orders of magnitude. Disk space increased by six or seven orders of magnitude. RAM capacity increased by six or seven orders of magnitude. Costs had fallen from dollars per bit to dollars per gigabit. The change in the hardware is hard to visualize, but just summing up all the orders of magnitude I mentioned leads us to about a thirty orders of magnitude increase in capability.
And all this in just over fifty years since Alan Turing’s lecture.
How many programmers are there now? How many lines of code have been written? How good is all that code?
Compare this timeline with the aeronautical timeline. Do you see the similarity? Do you see the gradual increase in theory, the rush and failure of the enthusiasts, the gradual increase in competence? The decades of not knowing what we were doing?
And now, with our society depending, for its very existence, on our skills, do we have the Sullenbergers whom our society needs? Have we groomed the programmers who understand their craft as deeply as today’s airline pilots understand theirs? Do we have the craftsmen whom we shall certainly require?
Craftsmanship is the state of knowing how to do something well and is the outcome of good tutelage and lots of experience. Until recently, the software industry had far too little of either. Programmers tended not to remain programmers for long, because they viewed programming as a steppingstone into management. This meant that there were few programmers who acquired enough experience to teach the craft to others. To make matters worse, the number of new programmers entering the field doubles every five years or so, keeping the ratio of experienced programmers far too low.
The result has been that most programmers never learn the disciplines, standards, and ethics that could define their craft. During their relatively brief career of writing code, they remain unapprenticed novices. And this, of course, means that much of the code produced by those inexperienced programmers is substandard, ill structured, insecure, buggy, and generally a mess.
In this book, I describe the standards, disciplines, and ethics that I believe every programmer should know and follow in order to gradually acquire the knowledge and skill that their craft truly requires.