Bob on Medical Device Software

Even though it has been over 6 months, my Zeo scam post is suddenly getting some comment traction. I thought I’d respond to some of these as well as clarify my thoughts.

I’m not really sure why Krunz thinks I’m an idiot.

First, I never said that changes in life style do not affect the quality of sleep. They do indeed. For example, for OSA (obstructive sleep apnea):

Some treatments involve lifestyle changes, such as avoiding alcohol and medications that relax the central nervous system (for example, sedatives and muscle relaxants), losing weight, and quitting smoking. Some people are helped by special pillows or devices that keep them from sleeping on their backs, or oral appliances to keep the airway open during sleep.

Second, I did say that because the ZQ score is based on sleep staging (no matter how crude), I can believe that an increased ZQ is indicative of better sleep.

My problem with the Zeo device is the claim that ZQ score improvement is caused by any particular life style change. This would be very difficult to validate.

Let’s say you recorded your ZQ score for 30 days without making any life style changes. There will be an inherent variability in the ZQ score that results from a variety of sources — electrode placement, user movement during sleep, etc.  Unless an introduced life style change can make a statistically significant change to the ZQ score you can not attribute causality to it. And even if there was a significant ZQ change you would still need to somehow prove that there were no other factors involved.

RobertF was not only more civil, but he took the time to detail his opinions and asked some good questions about mine. Here are my responses:

1. The red flag for me is when you make unvalidated claims. Anecdotal evidence of an improved ZQ score though “sleep coaching” is not validation.

The same is true for the “alarm clock” functionality. Zeo and others (e.g. Actigraphy) make similar claims about waking during lighter periods of sleep reducing sleep inertia.  Again, there is anecdotal evidence and even some testable theories, but a lot more research needs to be done in this area.

2. On one hand, Zeo does not claim that this is a therapeutic or a diagnostic device. From their web site:

The Zeo Personal Sleep Coach is neither a medical device nor a medical program and is not intended for the diagnosis or treatment of sleep disorders.

On the other hand they also say the Zeo provides:

…personalized sleep information and customized action steps to improve your Sleep Fitness™

Think about it. In one breath they say it isn’t, and then in the next they say it is!  It has nothing to do with FDA approval per se. It’s the contradictory claims that bother me.

3. I don’t think I was incredulous about this. I’ve mentioned several times that I think the dry sensor EEG technology probably provides enough signal quality to do a reasonable job of sleep staging.

4. It is the lack of “clinical validation” that is the most problematic for me. Basic sleep research may be heading in this direction — combining “sleep science, sleep education, neuroscience, behavioral psychology,…” — but they still have a long way to go.

5. Why would anybody that doesn’t have a sleep problem buy this product? If I did have sleep problems $350 would probably be well worth it, as long as it actually worked!  Sergey agrees.  If it didn’t work, I’d want my money back — and so would you.

6. You may not have any connection to the Zeo company, but the advisory board members are all paid to be there. I am not saying that this in any way lessens their scientific or professional credentials. On the contrary,  a good advisory board should be asking tough questions and doing their best to improve the product.

Wrap Up:

OK, I’ll admit it.  Maybe “scam” is too harsh. The reason I chose that term was because of its definition as a confidence trick:

an attempt to defraud a person or group by gaining their confidence

When I first read about the Zeo I felt that their presentation of the sleep science and technology was an attempt to gain a customer’s confidence. Beyond that there was little evidence that this product would help people.

I do not have any ill will towards Zeo. It appears that their customer service and return policies (30 day full refund) are good. As a medical device developer, I’m just pointing out what I think are important issues about this device.

I still maintain that the claims made by Zeo are misleading. You need to be able to show scientific evidence that the ZQ score actually does track with life style changes. Unless I’m missing something, Zeo has not done this.