Why is so much software so bad?

(Cross posted at my java.net blog)

Why is so much software so bad?

I've read or heard this question thousands of times, and I repeatedly ask this question myself (I am sometimes embarrassed to be a programmer when my non-techno-friends struggle with bad software). Robert Martin's blog entry on No Next Big Thing bemoans the sorry state of software quality, and prompted me to ponder this question again.

James Gosling's blog entry on Sharpening the Axe includes a confession that is a telling clue in our search for some of the causes for bad software. In Gosling's words:

"Often tool building is far more fun than actually doing the job at hand."

Many of the great programmers that I know are more passionately curious about building tools than about using tools. Actually doing the job at hand is sometimes an afterthought... and that can lead to bad (over-engineered/convoluted/overly complex) software.

Note: I will be using the word "tools" very broadly; "scaffolding" and "frameworks" are tools in the context of this blog entry.

Tools fall into two broad categories:

1. Tools that are built for the tool builder's use
   
2. Tools that are built for someone else to use
   

James' blog entry talks primarily about tools that he needs (or wants) to accomplish a specific task. Frequently these tools are disposable. It would be great if we honed these "I need it now" tools to a fine edge and kept them stored in our toolkits for future use, but that doesn't happen very often.

We tend to crank out "I need it now" tools with little thought towards "I'll need it again".

Corrective Behavior: When you find yourself tempted to build an "I need it now" tool, first ask yourself: "Will I need it again?"

If the answer is no, don't spent much time or effort on the tool.
Building any tool is a distraction from implementing the functionality that you've been asked to deliver. Bad software can result from spending too much time in preparation, and not enough in execution... it's a balancing act.

If the answer to the question "Will I need it again?" is yes, then you may be justified in spending more time and effort on the tool. I say "may be justified" because there are a million and one caveats to consider. If you will need a tool again, then someone else may have already created a tool that you could use for the same purpose. Because we like to build tools, we don't always check around to find out if a tool is already available.

Corrective Behavior: Before building a tool, look for a similar tool.

Actually, you should get into the habit of looking for tools long before you need them. Outside the world of software, I am a hardware store junkie. I love to browse the Harbor Freight catalog and prowl the aisles of Home Depot. My wife has learned not to roll her eyes when I rave about gadgets for driving screws at right angles or hanging a picture at a 32.7 degree angle. Most of the tools that I come across I will never need... but every so often I save my self a lot of time and effort because I knew that a tool existed to accomplish a specific task.

If you can't find a pre-existing tool and you do decide to build an "I'll need it again" tool, then you really have to start thinking about usability and reusability.

Tools that you will need again begin to resemble tools that you build for someone else to use. You have to start taking into account the learning curve needed to master the tool. You have to take into account the ways in which the tool can be misused. You have to start thinking about incorporating user feedback into your process for refining and maintaining the tool (With my fading memory, I won't remember building the tool, so when I need it in a few months I too will be a "first time user").

Corrective Behavior: Before finalizing a tool's design, explain to someone else how to use it.

The perceived quality of a tool has as much to do with how you use the tool (the tool's interface) as with how well it is built. You can craft a widget out of titanium alloy, but if the users can't figure out where to hold it they won't use it (and they'll probably tell others that you built a bad widget).

Many of the interfaces that we come up with for our tools are overly abstracted. We genericize our tools to the point of hyper-configurability. Configurability is great, but when we configure something wrong the result is often buggy software. Every layer of indirection between a requirement and the code that implements the requirement is an opportunity for introducing errors.

Much of the bad software out there could be due to layer upon layer of slightly mis-configured tools.

Corrective Behavior: When building a configurable tool, spend time to simplify the configuration process.

Recently XML configuration files have become all the rage to the point where I am beginning to wonder if it is possible to write and deploy a Java application without authoring an XML configuration file. It's always going to be a judgment call with regards to what should be configured via XML, what should be configured via annotations, and what should be configured via customized code. Whatever your choice, you must consider the likelihood of configuration errors and take steps to prevent, detect, and correct them. Often times this will require the creation of a supporting tool. Often time this leads to the cycle of building tools to build tools that build tools. If any of the tools is buggy, the end-user product may well be bad software.

At this point I need to back-track and loudly assert that the tool-building related factors that I have discussed are not the primary reasons that a lot of software is bad. Shoddy requirements and poor management are certainly bigger culprits in many instances... but we always need to look at ourselves as part of any solution.

While writing this blog entry, I started thinking about professional standards and a professional code of conduct. Using Google, I did a quick search on "code of conduct" and came up with a link to the following: ACMCode of Ethics and Professional Conduct.

The ACM's code is lofty and noble in it's tone, and seems to borrow a bit from the Hippocratic Oath.

I am going to digress from my tool-building-centric theme and lift my final Corrective Behavior directly from the ACM code:

"Quality professional work, especially in the computing profession, depends on professional reviewing and critiquing. Whenever appropriate, individual members should seek and utilize peer review as well as provide critical review of the work of others."

Sounds like a good topic for a future blog entry....