More Software Forensics and Why Analogies Suck

There's a recent article in the Baltimore Sun called Flaws in medical coding can kill which just rehashes static software  analysis (hat tip: FDA Trying to Crack Down on Software Errors).

I've discussed software forensics tools before. Yes, bad software has hurt and killed people, and there's no excuse for it.  I still don't think an expensive automated software tool is the silver bullet (which is implied by the article) for solving these problems.

But here's what really bugs me:

"If architects worked this way, they'd only be able to find flaws by building a building and then watching it fall down"

This is a prime example of why analogies suck.  The quote is supposed to somehow bolster the FDA's adoption of "new forensic technology". If you stop and think about it, it does just the opposite.

I guess you first have to consider the source --  a VP of Engineering for a forensic software vendor. This is exactly what a you'd expect to hear in a sales pitch.

What's truly ironic though is that a static analysis tool can only be used on source code! Think about it. Source code is the finished product of the software design and development process. Also, forensic science, by definition is the presentation of something that has already happened. It can only be done after the fact.

The logical conclusion you would draw from the analogy is that static analysis is probably useless because the building is already up!  If you step back and look at the full software quality process, this may well be true.

I'm not saying that static analysis tools don't have value. Like all of the other software tools we use, they have their place.

Just beware when you try to use an analogy to make a point.

UPDATE (7/5/08):

Here's another take on medical device bugs: When bugs really do matter: 22 years after the Therac 25.

UPDATE (7/16/08):
From Be Prepared: Software Forensics Gaining Steam at FDA, David Vogel of ­Intertech Engineering Associates says:

... that static tools are hyped to do more than they can actually deliver. “Static analysis looks for simple coding errors and does not apply heuristics to understand how it will perform dynamically because it is a static analysis tool”

I agree.

UPDATE (7/26/08):

Another reference : Are hospitals really safe?

UPDATE (9/16/08):

A couple more related articles:

Applying Static Analysis To Medical Device Software

Using static analysis to diagnose & prevent failures in safety-critical device designs

UPDATE (9/27/08):

Architecting Buildings and Software: Software architects are an important component in the creation of quality software and need to continue to refine and improve their role in the development process.  No matter how you try to bend and twist it though, the building analogy will always be problematic -- so why bother? Maybe that "intuitive understanding" of the construction industry just distracts us from being innovative about what's required to build software.

UPDATE (12/1/08): If Jeff wasn't a programmer he'd be a farmer: Tending Your Software Garden