The bottom line of the article Software Development Must Be Overhauled to Drive Growth in the Medical Device Industry is summed up by this:
The power and reach of converging IT trends means that business leaders need to understand the implications of a software-driven, connected-everything world.
The context of this statement is broad (“Technology Vision”), but it is still a driving force for the Medical Device industry. Certainly connected-everything, i.e. interoperability, has been an elusive goal (I’m being kind) for medical devices and data systems.
The Accenture report notes the following three Medical Device Industry market trends:
- Complexity of medical devices and how software has become a key differentiating success factor.
- Growing demand for mobile access to patient data and clinical systems.
- The importance and complexity of global markets when creating appropriate software for medical equipment products.
The use of Waterfall vs. Agile methodologies and other software development operating models (“Software Factories”, Outsourcing, etc.) is really dependent on an organization’s culture. The biggest challenge any company has in this regard is change. Even after the business recognizes the need to “restructure” operations, this is usually easier said than done.
A good example of Medical Device Open Source software is the West Health Institute Medical Device Interoperability project which is developing Standards-based embedded software. The OSS, Cloud, and “Module Care” sections only touched briefly on regulatory issues. For FDA regulated software there are significant design, risk, testing, and validation requirements that must taken into consideration when incorporating any 3rd party software. Further discussion is here: Open Source Medical Device Connectivity.
Just like many other industries, software is a critical component in Healthcare related products. I think most Medical Device companies already know this. Making software development more efficient (cost-effective) and flexible (to meet ever-changing market needs) while maintaining quality, performance, and security requirements is a difficult balancing act.
The Atlantic article The Robot Will See You Now is a good read. There’s a lot of discussion about how the AI technology developed for IBM Watson could benefit the health care system. Also, advanced computing combined with the use of smart phones for physiological data collection and transmission has real potential:
As sensors shrink and improve, they will increasingly allow health to be tracked constantly and discreetly—helping people to get over illnesses faster and more reliably—and in the best of cases, to avoid getting sick in the first place.
The preventative component of continuous monitoring could have a significant impact in the effectiveness of healthcare delivery.
The economics and psychology of “Health 2.0″ will be an evolutionary process. The movement towards less skilled healthcare professionals equipped with “clinical support” tools is inevitable. Hopefully that will be as close as we get to having to visit a robot doctor.
The latest incarnation of EEG-based devices comes from Muse – The Brainwave Sensing Headband.
Just like other BCI claims, How Mind-Controlled Games Work – And Why It’s Way, Way Bigger Than That is a new approach to consumer brain monitoring applications. From the Muse site (my highlighting):
Our early apps will be focused on building the core of your mind to improve intellectual skills such as memory and concentration, or emotional skills like maintaining composure in high stress situations. Other Muse apps would be just plain fun stuff so you could paint or compose music with your mind or play video games using your mind as the game controller.
The FAQ assures you they’re not mind reading and that it’s not a mind control device.
Taking the “brain heath” approach, see CES 2013: InteraXon debuts Muse along with Brain Health System application, is an interesting twist. I’m a big fan of EEG-based technology. The research efforts and advancements in the BCI field have the potential to improve many lives.
InteraXon is probably doing great things (e.g. the headband is very clever) and they appear to be active in the BCI community. My only issue is with the marketing claims being made. Just like the mind control game controllers that have come before (see Turning the Mind Into a Joystick), the reality of the current technology is still not able to live up to most people’s expectations. This seems especially true when it comes to something as subjective as concentration or stress. Also, painting with your mind — really?
InteraXon raised over $287,000 through Crowdfunding at Indiegogo: MUSE: The Brain-Sensing Headband that lets you control things with your mind. Many of the contributions levels included receiving a device and the brain fitness app. They also expect to provide developers with a SDK by mid-year. That might be fun to play with.
The FDA has announced the Medical Device Innovation Consortium (MDIC) which aims to help medical device companies get their products to market faster. See FDA, Private Groups Team Up to Speed Device Approval.
The term Regulatory Science is used 12 times on the single page MDIC Web site so it must be important. The FDA has been using regulator science in other health related areas since early 2010: see Advancing Regulatory Science.
This consortium is part of a much broader strategy (see the strategic plan) to improve both innovation and safety in FDA-regulated products. The MDIC site talks about subcommittees and projects but it’s unclear what specific medical device topics will be addressed. It will be interesting to follow their growth and progress.
BCI research is important work (see here). The availability of reasonably priced hardware and general purpose APIs has made it easy to investigate many aspects of how EEG processing of can be used to control the external environment.
The extrapolation of this work into the concept of mind reading software appears to be inevitable, but even after all these years, is still annoying. The latest incarnation of this is based on reputable work at Universities of Oxford and Geneva, and the University of California, Berkeley: Hackers backdoor the human brain, successfully extract sensitive data.
To start with, finding a correlation between P300 responses and a person’s image recognition — by 15% – 40% compared to random guessing — isn’t exactly earth shattering. Also, note that P300 is an average of multiple evoked responses. This requires many repetitions of the stimulus (16 times in this study) to reduce the noise enough and see the signal at all. As a practical matter, this is a really long way from brain malware.
Checkout Computers can read your mind. Still amazing!
Here’s the graphic from Hacking the Human Brain? Not As Impossible As You Think about the same research:
I didn’t realize there was a whole website for “NEWS ABOUT BRAIN-COMPUTER INTERFACES (BCI), MIND-CONTROLLED GADGETS & BIOFEEDBACK” — interesting stuff.
Microsoft has applied for a patent for using finger flexing to control a computer.
I’ve talked many times in the past about the use of EEG technology for computer control (brain-computer interface, BCI).
As discussed in the RWW article, there are many challenges to making this work. Just like with EEG, calibration of the EMG sensors and training will require innovative solutions.
It seems to me that this type of gesture-based control has quite a bit more potential than what can be obtained through the interpretation of EEG signals. In either case, the big benefit of advancements in these human-computer interface (HCI) technologies is that they could ultimately improve communications capabilities for the disabled.
The use of open-source software in medical devices has been a topic of discussion for many years. The Economist article Open-source medical devices: When code can kill or cure highlights continuing activity in the academic community and interest by the FDA in developing processes around its use.
One of the big unknowns in this area is how a community might be formed (Dreaming of Flexible, Simple, Sloppy, Tolerant in Healthcare IT has some thoughts on this).
From the article:
Eventually, medical devices might evolve into collections of specialised (and possibly proprietary) accessories, with the primary computing and safety features managed by an open-source hub.
This is in reference to both hardware and software, but in either case one major challenge will be how to incentivize contributions. Open-source means free to use and modify. If there is no financial gain to be had, other benefits for contributing need to be developed. Also, proprietary and open-source in the same sentence seems like an oxymoron, so I’m not sure how that’s going to work.
Another barrier would be liability risk. Let’s say you contributed software to this hub and that component ended up in a device that harmed or killed someone. All of the legal waivers, disclaimers, and releases in world wouldn’t necessarily keep you out of a court room.
I don’t think the basic arguments discussed in Open Source Medical Device Connectivity have changed a great deal. At the end of the day the medical device manufacturer will ultimately be responsible for ensuring the safety and efficacy of their device.
After close to two years of effort (see here) the Healthcare IT Stack Exchange Site is closing for good. HealthCare IT is closing:
it simply does not appear that this topic has a strong enough following on our network to support the site long-term
That says it all.
Healthy Architectures – Using CQRS and Event Sourcing for Electronic Medical Records presents a couple of interesting patterns for the management of healthcare data. The two patterns are:
- Command Query Responsibility Segregation (CQRS)
- Event Sourcing (ES)
Here’s another article that provides further clarification on the CQRS pattern and how it compares to ES and Task-Based UIs: CQRS, Task Based UIs, Event Sourcing agh!
These are high level design patterns that result in non-traditional and more complex system architectures when implemented. The healthcare domain is complex and alternate approaches for providing robust solutions are worth consideration.
My last discussion of Off-The-Shelf software validation only considered the high-level regulatory requirements. What I want to do now is dig deeper into the strategies for answering question #5:
How do you know it works?
This is the tough one. The other questions are important, but relative to #5, answering them is pretty easy. How to answer this question (i.e. accomplish this validation) is the source of a lot of confusion.
There are many business and technical considerations that go into the decision to use OTS or SOUP software as part of a medical device. Articles and books are available that include guidance and general OTS validation approaches. e.g. Off-the-Shelf Software: A Broader Picture (warning PDF) is very informative in this regard:
- Define business’ use of the system, ideally including use cases and explicit clarification of in-scope and out-of-scope functionality
- Determine validation deliverables set based on system type, system risk, project scope, and degree of system modification
- Review existing vendor system and validation documentation
- Devise strategy for validation that leverages vendor documentation/systems as applicable
- Create applicable system requirements specification and design documentation
- Generate requirements-traceable validation protocol and execute validation
- Put in place system use, administration, and maintenance procedures to ensure the system is used as intended and remains in a validated state
This is great stuff, but unfortunately it does not help you answer question #5 for a particular type of software. That’s what I want to try to do here.
OTS really implies Commercial off-the-shelf (COTS) software. The “commercial” component is important because it presumes that the software in question is a purchased product (typically in a “shrink-wrapped” package) that is designed, developed, and supported by a real company. You can presumably find out what design controls and quality systems are in place for the production of their software and incorporate these findings into your own OTS validation. If not, then the product is essentially SOUP (keep reading).
Contrast OTS with Software of Unknown Provenance (SOUP). It is very unlikely that you can determine how this software was developed, so it’s up to you to validate that it does what it’s supposed to do. In some instances this may be legacy custom software, but these days it probably means the integration of an open source program or library into your product.
This following list is by no means complete. It is only meant to provide some typical software categories and the strategies used for validating them. Some notes:
- I’ve included a Hazard Analysis section in each category because the amount of validation necessary is dependent on the level of concern.
- The example requirements are not comprehensive. I just wanted to give you a flavor for what is expected.
- Always remember, requirements must be testable. The test protocol has to include a pass/fail criteria for each requirement. This is QA 101, but is often forgotten.
- I have not included any example test protocol steps or reports. If you’re going to continue reading, you probably don’t need help in that area.
- Windows XP SP3
- Windows 7 32-bit and 64-bit
- Red Hat Linux
- Hazard Analysis: Do a full assessment of the risks associated with each OS.
Use your full product verification as proof that the OS meets the OTS requirements. This has validity since your product will probably only be using a small subset of the full capabilities of the OS. All of the other functionality that the OS provides would be out of scope for your product.
This means that a complete re-validation of your product is required for any OS updates.
There is no test protocol or report with this approach. The OS is considered validated when the product verification has been successfully completed.
- Pay particular attention to the hazards associated with device and device driver interactions.
- List all hazard mitigations.
- Provide a residual Level of Concern (LOC) assessment after mitigation — hopefully this will be negligible.
- If the residual LOC is major, then Special Documentation can still be provided to justify its use.
- Visual Studio .NET 2010 (C# or C++)
- Hazard Analysis:
For widely used compilers (like Microsoft products) full product verification can be used as proof of the OTS requirements.
Validation of a new compiler version , e.g. upgrading from VS 2008 to VS 2010: Showing that the same code base compiles and all Unit Tests pass in both can be used as proof. This assumes of course that the old version was previously validated.
The compiler is considered fit for use after the product verification has passed so there is also no test protocol or report in this case.
- For a vast majority of cases, I think it is safe to say that a compiler does not directly affect the functioning of the software or the integrity of the data. What a program does (or doesn’t do) depends on the source code, not on the compiled version of that code.
- The compiler is also not responsible for faults that may occur in devices it controls. The application just needs to be written so that it handles these conditions properly.
- For some embedded applications that use specialized hardware and an associated compiler, the above will not necessarily be true. All functionality of the compiler must be validated in these cases.
- Hazard Analysis: Both of these open source libraries are integrated into the product software. The impact on product functioning, in particular data integrity, must be fully assessed.
- You first list the requirements that you will be using. For example, typical logging functionality that might include:
For database functionality, listing basic CRUD requirements plus other specialized needs can be done in the same way.
I have found that the easiest way to test these kinds of requirements is to simply write unit tests that prove the library performs the desired functionality. The unit tests are essentially the protocol and a report showing that all asserts have passed is a great artifact.
- The logging system shall be able to post an entry labeled as INFO in a text file .
- The logging system shall be able to post an entry labeled as INFO in a LEVEL column of a SQL Server database.
- … same for ERROR, DEBUG, WARN, etc.
- The logging system shall include time/date and other process information formatted as “YYYY-MM-DD HH:MM:SS…” for each log entry.
- The logging system shall be able to log exceptions at all log levels, and include full stack traces.
Version Control Systems
- Hazard Analysis: These are configuration management tools and are not part of the product. As such, the level of concern is generally low.
- As above, you first list the specific functionality that you expect the VCS to perform. Here are some examples of the types of requirements that need to be tested:
You then write a protocol that tests each one. This would include detailed instructions on how to perform these operations along with the pass/fail criteria for each requirement.
- The product shall be able to add a directory to a repository.
- The product shall be able to add a file to a repository.
- The product shall be able to update a file in a repository.
- The product shall be able to retrieve the latest revision of files and directories.
- The product shall be able to branch a revision of files and directories.
- The product shall be able to merge branched files and directories.
Issue Tracking Tools
- Hazard Analysis: These tools are used for the management of the development project. Again, the level of concern is generally low.
- You only need to validate the functionality you intend to use. The features that you don’t use do not need to be tested.
- You simply need to test the specific functionality. Some example requirements — the roles, naming conventions, and workflow will of course depend on your organization and the tool being used:
A protocol with detailed instructions and pass/fail criteria is executed and reported on.
- A User shall be able to create a new issue.
- A User shall be able to comment on an issue.
- A Project Manager shall be able to assign an issue to a Developer.
- A Developer shall be able change the state of an issue to ‘ready for test’.
- A Tester shall be able to change the state of an issue to ‘verified’.
- The tool shall be able to send e-mail notifications when an issue has been modified.
- An Administrator shall be able to define a milestone.
Validation is a lot of work but is necessary to ensure that all of the tools and components used in the development of medical device software meet their intended functionality.