Bob on Medical Device Software

There’s a recent article in the Baltimore Sun called Flaws in medical coding can kill which just rehashes static software  analysis (hat tip: FDA Trying to Crack Down on Software Errors).

I’ve discussed software forensics tools before. Yes, bad software has hurt and killed people, and there’s no excuse for it.  I still don’t think an expensive automated software tool is the silver bullet (which is implied by the article) for solving these problems.

But here’s what really bugs me:

“If architects worked this way, they’d only be able to find flaws by building a building and then watching it fall down”

This is a prime example of why analogies suck.  The quote is supposed to somehow bolster the FDA’s adoption of “new forensic technology”. If you stop and think about it, it does just the opposite.

I guess you first have to consider the source –  a VP of Engineering for a forensic software vendor. This is exactly what a you’d expect to hear in a sales pitch.

What’s truly ironic though is that a static analysis tool can only be used on source code! Think about it. Source code is the finished product of the software design and development process. Also, forensic science, by definition is the presentation of something that has already happened. It can only be done after the fact.

The logical conclusion you would draw from the analogy is that static analysis is probably useless because the building is already up!  If you step back and look at the full software quality process, this may well be true.

I’m not saying that static analysis tools don’t have value. Like all of the other software tools we use, they have their place.

Just beware when you try to use an analogy to make a point.

UPDATE (7/5/08):

Here’s another take on medical device bugs: When bugs really do matter: 22 years after the Therac 25.

UPDATE (7/16/08):
From Be Prepared: Software Forensics Gaining Steam at FDA, David Vogel of ­Intertech Engineering Associates says:

… that static tools are hyped to do more than they can actually deliver. “Static analysis looks for simple coding errors and does not apply heuristics to understand how it will perform dynamically because it is a static analysis tool”

I agree.

Pete Gordon asked a couple of questions regarding FDA regulations for Internet-based reporting software that interface with medical devices. The questions are essentially:

1. How much documentation (SRS, SDS, Test Plan) is required and at what stage can you provide the documentation?
2. How does the FDA view SaaS architectures?

The type of software you’re talking about has no real FDA regulatory over site. The FDA has recently proposed new rules for connectivity software. I’ve commented on the MDDS rules, but Tim has a complete overview here: FDA Issues New MDDS Rule. As Tim notes, if the FDA puts the MDDS rules into place and becomes more aggressive about regulation, many software vendors that provide medical devices interfaces will be required to submit 510(k) premarket approvals.

Dealing with the safety and effectiveness of medical devices in complex networked environments is on the horizon. IEC 80001 (and here) is a proposed process for applying risk management to enterprise networks incorporating medical devices.  My mantra: High quality software and well tested systems will always be the best way to mitigate risk.

Until something changes, the answer to question #1 is that if your software is not a medical device, you don’t need to even deal with the FDA. The answer to question #2 is the same. The FDA doesn’t know anything about SaaS architectures unless it’s submitted as part of a medical device 510(k).

I thought I’d take a more detailed look at the architecture we’re talking about so we can explore some of the issues that need to be addressed when implementing this type of functionality.

This is a simplified view of the way medical devices typically interface to the outside world. The Communications Server transmits and receives data from one or more medical devices via a proprietary protocol over whatever media the device supports (e.g. TCP/IP, USB, RS-232, etc.).

In addition to having local storage for test data, the server could pass data directly to an EMR system via HL7 or provide reporting services via HTTP to a Web client.

There are many other useful functions that external software systems can provide. By definition though, a MDDS does not do any real-time patient monitoring or alarm generation.

Now let’s look at what needs to be controlled and verified under these circumstances.

1. Communications interaction with proper medical device operation.
2. Device communications protocol and security.
3. Server database storage and retrieval.
4. Server security and user authentication.
5. Client/server protocol and security.
6. Client data transformation and presentation to the user (including printed reports).
7. Data export to others formats (XML, CSV, etc.).
8. Client HIPAA requirements.

Not only is the list long, but these systems involve the combination of custom written software (in multiple languages), multiple operating systems, configurable off-the-shelf software applications, and integrated commercial and open source libraries and frameworks. Also, all testing tools (hardware and software) must be fully validated.

One of the more daunting verification tasks is identifying all of the possible paths that data can take as it flows from one system to the next. Once identified, each path must be tested for data accuracy and integrity as it’s reformatted for different purposes, communications reliability, and security. Even a modest one-way store-and-forward system can end up with a hundred or more unique data paths.

A full set of requirements, specifications, and verification and validation test plans and procedures would need to be in place and fully executed for all of this functionality in order to satisfy the FDA Class II GMP requirements. This means that all of the software and systems must be complete and under revision control. There is no “implementation independent” scenario that will meet the GMP requirements.

It’s no wonder that most MDDS vendors (like EMR companies) don’t want to have to deal with this. Even for companies that already have good software quality practices in place, raising the bar up to meet FDA quality compliance standards would still be a significant organizational commitment and investment.

Many people still confuse verification (was the product built right?) and validation (was the right product built?). The benefits of both of these activities are well covered in Software Validation: Turning Concepts into Business Benefits:

Software V&V is a FDA requirement, but the same methodologies can be used to improve non-device software as well.

First a definition:

driv·en –adjective

1. : having a compulsive or urgent quality
2. : propelled or motivated by something — used in combination <results-driven>

Driven software development methodologies abound:

• Behavior-Driven Development (BDD)
• Design-Driven Development (D3)
• Domain-Driven Design (DDD)
• Feature-Driven Development (FDD)
• Model-Driven Architecture (MDA)
• Process-Driven Development (PDD)
• Test-Driven Development (TDD)

Many of these are encompassed by the iterative Agile software development methodologies. Collectively they are sometimes referred to as the XDD acronyms. As you might expect, these along with all of the other competing, contrasting, and overlapping development philosophies can cause a software developer much consternation. Confessions of a Software Developer* is a good example of the overload that can occur.

My reason for bringing up driven methodologies is not to complain about being overwhelmed by them (which, like most others, I am). It’s simply to point out the contradiction of X-Driven with the Merriam-Webster definition. I think this will help us better understand what should really be driving us.

Look closely at definition #2. Propelled or motivated by something … results-driven. What is that something? Ah ha!

The fundamental motivation for all of these development approaches is to:

Improve productivity and quality.

This is the result, the goal. Behavior, Model, Test, etc. are all just the means by which we are trying to achieve this desired result. It’s the result that we’re driven towards, not the methods and techniques we use to get there.

So, in order to make this distinction clear and to eliminate confusion in the future, I propose that all these methodologies be renamed from Driven to Guided. Think of them like you would a GPS system in your car, except these will allow you to find software Nirvana. TDD is now TGD, and the whole lot is known as XGD.

The point here is that you should not let any particular development philosophy blind you to what the real purpose of using it is in the first place. Being guided by a methodology helps me remember that better than when I’m driven by it. Also, the whole concept of being driven seems exclusionary to me. You shouldn’t hesitate use the pieces and parts of any combination of these techniques that best suites your needs.

We tend to focus a lot of attention on tools and methodologies for improving software quality. I thought it would be worth while taking a step back to try to understand what the root causes of software defects are. Fortunately there has been decades of research that have analyzed the most common sources of software defects.

After also looking at some related development sins, I’ll summarize what this new understanding means to me as a software developer.

An often sited article in IEEE Computer is Software Defect Reduction Top-10 List (Vol. 34, Issue 1, January 2001, 135-137) . Here’s a summary (from Software Engineering: Barry W. Boehm’s Lifetime Contributions to Software Development, Management, and Research):

1. Developers take 100 times less effort to find and fix a problem than one reported by a customer.
2. Half of software project work is wasted on unnecessary rework.
3. Twenty percent of the defects account for 80% of the rework.
4. Twenty percent of modules account for 80% of the defects and half the modules have no defects.
5. Ninety percent of the downtime comes from 10% of the defects.
6. Peer reviews catch 60% of the defects.
7. Directed reviews are 35% more effective than nondirected ones.
8. Discipline can reduce defects by 75%.
9. High-dependability modules cost twice as much to produce as low-dependability ones.
10. Half of all user programs contain nontrivial defects.

This list is based on empirical research and is a good starting point for understanding how to avoid predictable pitfalls in the software development process.

A broader perspective is provided by Pursue Better Software, Not Absolution for Defective Products — Avoiding the “Four Deadly Sins of Software Development” Is a Good Start. Here are the four deadly sins:

The First Deadly Sin: Sloth — Who Needs Discipline?

The Second Deadly Sin: Complacency — The World Will Cooperate with My Expectations.

The Third Deadly Sin: Meagerness — Who Needs an Architecture?

The Fourth Deadly Sin: Ignorance — What I Don’t Know Doesn’t Matter.

The SEI article concludes:

We believe that the practice of software engineering is sufficiently mature to enable the routine production of near-zero-defect software.

How can you not smile (or even LOL) at that statement? Despite that, I like the reduction of the problem into its most basic elements: human shortcomings. That’s why the conclusion is so preposterous — software development is a human activity, and a complex one at that. You’re trying to produce a high quality software solution that meets customer expectations, which is a difficult thing to do.

Another list of software development sins can be found in The 7 Deadly Sins of Software Development.
#1 - Overengineering (in complexity and/or performance)
#2 - Not considering the code’s readership
#3 - Assuming your code works
#4 - Using the wrong tool for the job
#5 - Excessive code pride
#6 - Failing to acknowledge weaknesses
#7 - Speaking with an accent (naming conventions)

There are some tool/language specific items here, but this list generally follows the same trend of discovering typical developer shortcomings that can be avoided.

Another source of software defects is poor project planning. More sins (deadly again) can be found in the Steve McConnell article: Nine Deadly Sins of Project Planning.

It’s pretty easy to see from these categorizations where a lot of the software development and management techniques, tools, and practices came from. As you might have expected, many are focused on human behavior and communication as a key component for improving software quality. For example, take the Agile Manifesto:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

This vision is very telling about what the manifesto writers considered to be a primary cause of software defects.

Another perspective is Fred Brooks’ famous 1986 ‘No Silver Bullet‘ paper (also see here) that distinguishes “accidental” repetitive tasks from “essential” tasks. From the article:

There is no single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity.

Even after twenty years of significant progress in software engineering, I believe that this is still a true statement.

Conclusion:

There are many complex factors that contribute to software defects. There is clearly no one-size-fits-all solution. As a developer, this means that I have to:

1. Be aware of my own shortcomings and biases.
2. Continually try to improve my development, communication, and management skills.
3. Understand how each new tool or methodology that claims to improve software quality fits into the bigger picture of what I (both personally and as an organization) am trying to accomplish.

FMEA (Failure Mode and Effects Analysis) is a regular part of our development process (we call it “Hazard Analysis”), but I was unfamiliar with FRACAS (Failure Reporting, Analysis, and Corrective Action Systems) until I ran across this: 10/21/2007 FRACAS? – Never heard of it.

I’d also never heard of “software reliability growth”. The model is described here, and there are also some other good links available from the Google search.

I’m a software developer, not a quality systems person. According to Jan, even medical device quality people are not that familiar with these methods. In addition to Jan’s explanation for this, one reason I (or others) might not have heard of these is that my experience has been exclusively in Class II non-invasive diagnostic devices. The development of Class III life support and implantable devices is a whole different animal when in comes to quality control rigor.

CAPA’s (Corrective and Preventive Action) are of course part of the standard FDA Quality System Regulations (§ 820.100). These procedures are primarily implemented to deal with quality problems after the product has been released to the field.

The concept of preventing recurrence of observed errors (FRACAS) during the development process is certainly an interesting one. The difference between CAPA and FRACAS is similar to the argument I made regarding Software Forensics — these techniques should be used to ensure quality before the product is released.

“The Gray Sheet” has an article called CDRH Software Forensics Lab: Applying Rocket Science To Device Analysis. Can it really be true that CDRH is doing static code analysis for detecting software defects in recall investigations?

Static code analysis has been around for a long time. I remember using Lint back in the old days. Nowadays all commonly used computer languages (like C and C++) have fairly advanced analysis tools. For Microsoft .NET languages there are tools like FxCop and ReShaper. These tools are great for spotting trouble areas and maintaining code conventions during the development process.

I just have a hard time imagining a process using these tools that would successfully detect real defects in complex medical device software. Especially software that’s controlling hardware devices, doing communications tasks, and recording sensor data. Also, what about all of the assembly language code for embedded processors and DSPs?

Anyway, from the article:

A recent review in the forensics lab found 180 “questionable constructs” in 100,000 lines of code, but only two turned out to be real design issues, Taylor said.

He also pointed to two other cases where static analysis of the software did not find any bugs, thus clearing software as a root cause candidate in the recall investigations.

These statements beg the following questions:

1. Were the two “real design issues” related at all to the device failures?
2. Just because no static analysis “bugs” were found, why does that exonerate the software from being the cause of the failure?

I’m not impressed that static analysis has “been embraced by the aeronautical and automotive sectors”. IMHO this approach just seems like it would create a lot of work for very little return. The manufacturer that produced the device should be responsible for tracking down and fixing the software (and/or hardware) defects.

I’ll stop now.

UPDATE (25-Oct-07):

Check out Tim’s post on this subject: FDA Raises Bar on Medical Device Software Testing.

I ran across this post: Why Making Software Companies Liable Will Not Improve Security. It’s a rather long piece that discusses the liability of software makers for security breaches. In the middle of the article he talks about his experience working on FDA regulated medical device software. I think his depiction is a little harsh, but probably not that far off depending on the environment you’re working in. The conclusion from his FDA experience is:

In short, I believe that any attempt to impose quality on software by inspection and rule books is doomed to failure.

I would say that there are no single set of rules that can ensure software quality (that’s not quite ‘doomed to failure’, but may be close). I think the FDA “rule book” (as I briefly describe here) is a full product life cycle quality system that generally meets its intended purpose. It doesn’t ensure that all medical device software is free from defect. Far from it. The regulations simply provide the FDA with a means for determining what the product requirements and design were and how well it was actually tested. It’s up to the regulators to use that information for deciding whether a product meets their quality standards.

On a side note, the post talked about how in 1997 50% of FDA device recalls were due to design defects. The article Failure Modes in Medical Device Software analyzes software-only FDA recalls from 1983-1997 and is a good read on the breakdown of software defects. According to that article, only about 10% of the total FDA recalls (1994-1996) were software related. It would be interesting to know how that number has changed since.