This article popped up on reddit and news.ycombinator.com. See the Romanian version, Russian version, Estonian version, Portuguese (none of which I can verify say the same thing).

While my talent lies in software, my graduate studies were in computer engineering (designing and building digital computers). One observation always struck me: computer engineering seemed more straightforward than computer science (building software). There are a set of engineering design rules to follow and engineering projects are much more likely to work out than software projects. Yes, there are some spectacular engineering failures but, empirically, reliable and useful software is more difficult to pull off. To illustrate my point, consider the amazing space probes we send throughout the solar system. The spacecraft themselves work extremely well, often living long past their design lifetimes. Their control software, on the other hand, has to be constantly tweaked and patched in flight to keep the mission together. I am not trying to say that engineering is easy; it is not. My goal is simply to explain why software is harder to get right than physical construction projects in general and why "software engineering" is an inappropriate term.

The Economist magazine (Nov 27, 2004 p. 71) cites the Standish Group's estimates that

...30% of all software projects are canceled, nearly half come in over budget, 60% are considered failures by the organizations that initiated them, and 9 out of 10 come in late.

The article goes on to point out that while few large infrastructure projects are completed on time and on budget, you usually get something in the end. You have to enter the realm of software (or the military) in order to spend billions and get nothing for it. The biggest reason for this profligate waste lies in that, while a partially completed building or infrastructure project with flaws can still be useful, software either works or it doesn't. Because software is devilishly difficult to get right, we often end up with nothing (e.g., FBI Dumps Botched Computer Overhaul, F.B.I. Faces New Setback in Computer Overhaul).

Why is writing software not like engineering? The answer lies in a single fundamental difference with far-reaching ramifications: engineering is constrained by the real, physical world and software is not. While obvious, this is the crucial difference that explains why software development is harder to get right. The next few sections explore these ramifications.

Engineering components exist in the real world

Physical engineering projects are easier to visualize and, therefore, easier to build; heck, you can actually touch the components and the final product with your hands. The further you get away from the Newtonian view of the physical world that we experience in everyday life, the more difficult the project. Quantum physics is difficult because particles don't behave like rocks. String theory, the most promising method of modeling quantum physics, is primarily difficult to understand because it deals with six or seven extra dimensions beyond the usual three spatial dimensions. Imagine trying to design and build a house on a Calabi-Yau Manifold (image to the right is a 3-D projection).

Software does not even have the concept of dimension. You might say that there are no lines within which to color. At least for engineering projects above the quantum level, no one expects to find structures that "defy the laws of physics." This limits what you can physically build and how you can build it but at least it provides a well-defined world in which to work. The ethereal world of software does not have this luxury, but our tools are getting better. Object-oriented programming was invented to make writing software more familiar to our hunter gatherer brains; that is, to make software components with properties and behavior like objects in the real world.

Engineering components interact in more predictable ways

Engineering is less risky than software because engineering experiences fewer constituent component interactions. While minor changes to one part of a car's structure can easily affect the crash robustness of another, it would be unusual for a design flaw in the dome light to cause intermittent engine stalls. In a home construction project, you would have to work pretty hard to make a toilet flush every time someone rang the doorbell.

When building software, on the other hand, you have to be hypervigilant to avoid these undesirable interactions. One of the reasons many programmers like pure functional programming is its lack of side effects--there simply is no way for the toilet to flush inadvertently when the doorbell rings.

The situation is even worse in languages like C++ without inherent buffer overflow protection. A remote fragment of code can unintentionally alter the behavior of the overall application much more easily. In fact, it is this particular weakness that most hackers exploit. By causing a buffer overflow, an attacker can force the program to open a hole in its defenses.

Engineering has fewer fundamental midcourse design changes

The final reason that software is more difficult to get right than engineering projects concerns midcourse design changes. The physical world is not as malleable as the insubstantial world of software and, consequently, clients simply have lower expectations. Congress does not go to NASA halfway through a moonshot and ask them to go to Mars instead. Most engineering projects are able to actually use the waterfall design method: determine functional requirements, design, implement, test. For most software projects, this is a recipe for disaster.

Unfortunately, midcourse design changes happen in software all the time--every time the customer gets a peek at the running software. In fact, the agile software development method is a direct response to ever-changing design requirements. Programmers are asked to embrace change as an opportunity. Still, constant changes to the design hamper development efforts and developers must constantly refactor software to prevent it from becoming a tangled, unmaintainable mess.

You might ask why, if software is so hard to get right, airplanes do not fall out of the sky. It turns out they do occasionally and bad software is often the culprit; e.g., watch a fly-by-wire Airbus 320 crash. Most of the time, however, hardware control software for airplanes, cars, and medical devices works as expected. The reason for the increased reliability is threefold. First, there are lives at stake and people are likely more careful. Secondly, the software developers are not asked to alter the software fundamentally during development. For example, a propeller driven airplane does not become a jet aircraft in the middle of development. An engineering team would start over, whereas a software team is typically asked to make radical changes midstream. And, finally, such software deals with controlling physical devices and starts to gain some of the advantages afforded physical engineering projects.

Is software development a science?

So if software development is not like engineering, what is it? We call the discipline computer science, but I am not sure that term is completely appropriate either. You may recall the old joke, "If a discipline has 'science' in the title, it probably isn't." In my view, science is about discovering and describing physical phenomena using a scientific method. Merriam-Webster describes the scientific method as:

Principles and procedures for the systematic pursuit of knowledge involving the recognition and formulation of a problem, the collection of data through observation and experiment, and the formulation and testing of hypotheses.

That description smacks of debugging more than the act of writing software. Computer science is about building things like engineering but without the luxury of a toolbox and components taken from the physical world. No one has worked out reliable and effective procedures for building large pieces of software as the engineers have done for physical project. As Alan Kay puts it, software is "engineering of a sort...but just done by brute force". A conversation with Alan Kay:

If you look at software today, through the lens of the history of engineering, it's certainly engineering of a sort--but it's the kind of engineering that people without the concept of the arch did. Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves.

Writing software is more an art than an engineering discipline

Writing software is most similar to writing fiction novels. Writing novels is also an act of creation in an unconstrained and ethereal medium with few well-established construction rules. We know good writing when we see it, but it is hard to teach. Experience writing and feedback from better writers (coders) is the most reliable means of becoming a good writer (coder). Without a well understood process, software will remain more an art than a science. The term "software engineering" is more a goal than how we actually write software.