Bob on Medical Device Software

In today’s New York Times business section there’s a piece called: Moving Mountains With the Brain, Not a Joystick. I’ve previously discussed both of the mentioned EEG-based headsets here.The article highlights some of the problems that this type of technology will face in the consumer marketplace:

“Not all people are able to display the mental activity necessary to move an object on a screen,” he said. “Some people may not be able to imagine movement in a way that EEG can detect.”

I agree. Even though Emotiv claims that “all 200 testers of the headset had indeed been able to move on-screen objects mentally” it’s very doubtful that the device will have that level of success (100%!) with real gamers.They talk about the use of facial muscle activity (EMG) in addition to the EEG signal. With proper electrode placement, I think EMG holds far more promise for enhancing the gaming experience. Even EOG could be used effectively as a feedback and control mechanism. Reliable EEG processing for this purpose is still a long way off.

UPDATE (6/17/08): More of the same: No Paralysis in Second Life

UPDATE (6/29/08): Here’s a pretty good description of how these devices are being used for control purposes:  OCZ’s Neural Impulse Actuator (The flying car of control schemes).

UPDATE (7/21/08): An even more thorough evaluation: OCZ NIA Brain-Computer Interface. A generally positive and realistic assessment of the technology:

…the NIA isn’t a replacement for traditional input methods, it is merely a powerful supplement.

I guess I’m a sucker for EEG related technology (see all my HCI posts). So when I run across an article like A baseball cap that reads your mind I can’t help but comment on it.

Unlike other “mind reading” systems that make unrealistic claims, I can see this research and wireless technology leading to something quite useful. The ability to discern closed eyes and drowsiness by the presence of alpha waves (8-12 Hz) in human EEG is well known.

Developing an affordable product that provides a timely audible alert to a driver that’s about to fall asleep could have a huge impact. From (beware, this is a PowerPoint presentation) Fatigue and Automobile Accident Risk:

The US Department of Transportation estimates that 100,000 accidents reported are due to drowsiness and/or fatigue. These crashes result in 1550 deaths annually (4% of traffic fatalities) and $12.5 billion in monetary losses.

Even the annoyance of false alerts would be worth the lives saved. And of course it’s convenient that a lot of truck drivers already wear baseball caps.

The New York Times had a couple of articles over the last few days that deal with short-term memory loss.

David Brooks’ 11-Apr-08 commentary called The Great Forgetting summarizes the Bad Memory Century best with:

In the era of an aging population, memory is the new sex.

In addition to taxes and death, aging is something that none of us can avoid. I was born in the later part of the baby boom, so over the last few years I have become acutely aware of these short-term memory challenges. Remembering to write down thoughts and lists has become essential. And I’m still young, relatively speaking anyway. Like the other inevitables, you never think it’s going to happen to you. But it does!

The 13-Apr-08 Sunday Magazine’s Idea Lab Total Recall speculates about embedding a computer chip in the brain in order to improve short-term memory. I know that short-term memory loss is a problem, but who would have guessed that “sky divers have been known to forget to pull their ripcords — accounting, by one estimate, for approximately 6 percent of sky-diving fatalities.” !!

Interestingly, the brain is a particularly effective associative memory system:

…, studies suggest that if you learn a word while you happen to be slouching, you’ll be better able to remember that word at a later time if you are slouching than if you happen to be standing upright.

Neural prosthetics are a long way off (see here), but the concept of embedding a Google search engine in your brain is certainly intriguing — and scary to most.

In the mean time, you can follow the suggestions in How to Cope With Short Term Memory Problems.

The human-computer interface (HCI) will continue to be a major challenge for the future. The iPoint Presenter is an approach that makes a lot of sense. It’s been depicted as the future of computer interaction in movies like Minority Report and could easily be imagined as the next generation Wii.

Unlike the EEG-based “mind reading” devices that I’ve discussed before, this technology could be made affordable and reliable, so it holds much more promise. Plus that, it’s very cool.

UPDATE (3/5/08):

This is HCI related anyway: University of Bremen’s Brain-Computer Interface: The future world is here. This is an interesting approach for helping the disabled. LEDs are flashed at specific frequencies which causes the visual cortex to respond in a corresponding manner. When the person looks at that one LED or another the EEG response is detected and initiates the desired activity or makes the associated selection (e.g. letters or numbers).  The communication rate is slow, but this is a realistic technique nevertheless.

Via Slashdot, Emotiv has a new Brain control headset for gamers scheduled to come out later this year:

• Sensors respond to the electrical impulses behind different thoughts; enabling a user’s brain to influence gameplay directly
• Conscious thoughts, facial expressions, and non-conscious emotions can all be detected
• Gyroscope enables a cursor or camera to be controlled by head movements
• The headset uses wi-fi to connect to a computer

I’ve discussed this type of Mind Reading Software before (and here).

Interpreting motion and/or motor signals (facial expressions) is one thing. I’d love to know what type of EEG processing will be used to detect conscious thoughts and non-conscious emotions. At the very least, this type of quantitative EEG analysis has to begin with high quality EEG signals. I don’t know how this can be done from an unprepared scalp and electrodes that are applied without gel.

UPDATE (3/2/08):

Via Slashdot, here (and here) is another game controller from OCZ (the Neural Impulse Actuator (NIA) is not currently a listed product). The NIA works by “… reading bioptentials. These include activities of the brain, the autonomous nervous system and muscle.” You can’t help but be skeptical about the value of this technology for these purposes.

UPDATE (3/22/08):

More information on the Emotiv device: ‘Mind Gaming’ Could Enter Market This Year.

I ran across Commercial brain computer systems are coming today (original article is here). This is a cool graphic:

I’ve talked previously about Human-Computer interface (HCI) work.

Invasive BCI technologies have incredible potential. In addition to the unique device-human interface challenges (primarily sensors), it will also require significant advancements in the basic understanding of the underlying electrophysiology (cortical and motor). As reported, EEG-based processing will continue to be a corner-stone for this work, just as it is for non-invasive systems.

This slide from the report (pp. 170) summarizes the uses and importance of this type of work:

Here’s a shorter overview of BMI: Mind Controlled Bionic Limbs

I spent quite a few years developing diagnostic Electroencephalography (EEG) systems and software. I always get a kick out of articles with titles like this one: Microsoft Working On Mind-Reading Software. It’s the mind-reading part that gets me because your first impression is that Microsoft is developing technology that will allow it to somehow detect what you’re thinking. This, of course, will not be happening in the near or foreseeable future.

The work that Microsoft Research is doing in this area (see here) is fundamental research on the Human-Computer Interface (HCI). The Using a Low-Cost EEG for Task Classification in HCI Research article uses standard frequency domain EEG features (delta, theta, alpha, etc.) as classifiers in a Bayesian Network for differentiating three mental tasks. What was interesting to me was that they recognized the limitations of using EEG technology alone as a human-computer interface. The understanding and use of other physiological data (e.g. motor activity) along with EEG will have to be explored as a way to improve task detection.

Not only is this type of work important for meeting the needs of the physically disabled, as the Wii and Surface have shown, innovative HCI systems can have a dramatic affect on how we all interact with computers.

‘Thought-reading’ system controls wheelchair and synthesizes speech is another one. The system processes larynx nerve signals for speech synthesis and wheelchair control. The technology looks very cool and has the potential to improve the lives of handicapped individuals. I suppose you could consider motor neuron activity as the output of thought, but ‘thought-reading’ just feels like a misnomer. Maybe it’s just me.

Another ‘mind-reading’ technique is the use of Evoked Potentials (EP). One that got a lot of press is a few years back was Brain Fingerprinting (also see here). I’m sure there’s still on-going research in the P300 area, but nothing has grabbed much attention since.

Also, checkout Computers can read your mind. Amazing!

UPDATE:

I found some companies that appear to be trying to use EEG processing algorithms for HCI. Both are focused on the gaming industry. They provide no details on how their products work, so it’s hard not to be skeptical about their functionality claims.

NeuroSky

Emotiv

Also, Smart BrainGames provides more classical biofeedback development systems. All are mentioned here.

UPDATE-2:

Here’s another interesting technology: Functional near-infrared spectroscopy (fNIRS) is an emerging non-invasive, lightweight imaging tool which can measure blood oxygenation levels in the brain. Check out the article here.

UPDATE-3 (15-Oct-2007):

Here’s the Microsoft patent application: Using electroencephalograph signals for task classification and activity recognition (via here).

UPDATE-4 (13-Nov-2007):

Check out Brain2Robot Project which uses EEG signal processing (my highlighting):

Highly efficient algorithms analyze these signals using a self-learning technique. The software is capable of detecting changes in brain activity that take place even before a movement is carried out. It can recognize and distinguish between the patterns of signals that correspond to an intention to raise the left or right hand, and extract them from the pulses being fired by millions of other neurons in the brain. These neural signal patterns are then converted into control instructions for the computer.

If they can do this reliably, that’s quite an accomplishment.