Archive for the 'Technology' Category

Deep Learning

deepLearningAI500I recently attended a Deep Learning (DL) meetup hosted by Nervana Systems. Deep learning is essentially a technique that allows machines to interpret sensory data. DL attempts to classify unstructured data (e.g. images or speech) by mimicking the way the brain does so with the use of artificial neural networks (ANN).

A more formal definition of deep learning is:

DL is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using multiple processing layers with complex structures,

I like the description from Watson Adds Deep Learning to Its Repertoire:

Deep learning involves training a computer to recognize often complex and abstract patterns by feeding large amounts of data through successive networks of artificial neurons, and refining the way those networks respond to the input.

This article also presents some of the DL challenges and the importance of its integration with other AI technologies.

From a programming perspective constructing, training, and testing DL systems starts with assembling ANN layers.

For example, categorization of images is typically done with Convolution Neural Networks (CNNs, see Introduction to Convolution Neural Networks). The general approach is shown here:

Construction of a similar network using the neon framework looks something like this:

Properly training an ANN involves processing very large quantities of data. Because of this, most frameworks (see below) utilize GPU hardware acceleration. Most use the NVIDIA CUDA Toolkit.

Each application of DL (e.g. image classification, speech recognition, video parsing, big data, etc.) have their own idiosyncrasies that are the subject of extensive research at many universities. And of course large companies are leveraging machine intelligence for commercial purposes (Siri, Cortana, self-driving cars).

Popular DL/ANN frameworks include:

Many good DL resources are available at: Deep Learning.

Here’s a good introduction: Deep Learning: An MIT Press book in preparation

The Bumpy Road to a New Development Laptop

My 6 year old Lenovo T400 finally gave up the ghost. It didn’t totally die (it probably never will, thank you IBM), but the screen was starting to flicker and it reliably rebooted itself whenever I was doing something useful. Very annoying.


I went though the standard 5 stages of grief:

  1. Denial: All T400’s do this.
  2. Anger: “Damn it, why does this thing keep crashing? I’m sick of this sh*t!”.
  3. Bargaining: Maybe if I update to 14.04 it will stop doing this.
  4. Depression: “This sucks!”
  5. Acceptance: OK, time to buy a new laptop.

I’m fine now, but that was a rough 30 minutes!

Decision Process

I’m not going to detail all of my system requirements or decision making process, but here’s a high level outline:

  • I primarily need a Ubuntu development machine. My T400 is a dual boot 12.04/XP.  In recent years I’ve rarely used Windows, but there are some tools that are nice to have around (e.g. Visual Studio).
  • I looked hard at the MacBook Pro but at the end of the day I just couldn’t bring myself to go that route. Besides the higher hardware cost/performance ratio re: the alternatives, I guess I’m just not a Mac person.
  • I really wanted to get an Ultrabook form factor. Not only for the portability, but I’m not ashamed to say that the ‘cool factor’ played a part in the decision.
  • I looked at all of the standard Ultrabook offerings: Lenovo, ASUS, Dell, System76, Acer, etc. No touch, no ‘convertible’ (if you need a tablet, buy a tablet), no Windows 8. The deciding factor for me was reliability. Besides the T400, I have a T60 in the closet that still runs fine.
  • So Lenovo it is. The history of the X1 Carbon (see The 2014 Lenovo X1 Carbon: Lenovo Giveth, And Lenovo Taketh Away and Lenovo ThinkPad X1 Carbon 2014 review)  goes back to 2011. The latest version (2014, or Carbon 2) has taken a lot of heat over the keyboard, function keys, and trackpad changes. I’m sure these opinions have merit, but I just want a fast machine that works!

Buying Experience (not good!)

Beware of the Lenovo Outlet. I purchased a ‘ThinkPad X1 Carbon 2 – New’:


Here’s the condition definition (my highlight):

Products that are discontinued, overstocked, or returned unopened. These items are in their original factory sealed packaging and have never been used or opened.

Boy was I disappointed when the package arrived! First, the only thing in the box was the laptop. No AC power adapter, no docs, no nothing. To my amazement, the machine was in suspend mode. When I opened the lid it came out of hibernation to a Win7 user password prompt! I didn’t even try to guess a password. I couldn’t believe it!

The machine was in pretty good shape physically, a little dirty and missing a foot pad, but no dents or scratches. Certainly opened and used!  At least the BIOS confirmed that I got the correct hardware (i7, 8G RAM, 256G SSD).

After many calls to multiple Lenovo service centers I got nowhere. No return, no exchange. Maybe I should write a letter to The Haggler, but even then I probably wouldn’t return the machine anyway. I got a great price (much better than what I could find on eBay) and the Lenovo Outlet no longer has any i7 X1 Carbon’s listed.  Also, I’m a techie so disk partitioning and re-installed OS’s is not a problem.

I’m thinking now that Lenovo might have screwed up a repair shipment and I ended up wiping some poor schmuck’s SSD. Oh well.

Anyway, as unpleasant as this was, I now have a development laptop that should meet my needs for many years to come.

Installation Notes

  • Dual boot. Here’s the right way: WindowsDualBoot, but because I installed Ubuntu first (mistake) here’s what I did:
    1. Used GParted  to partition the disk to my liking. Don’t forget to add a Linux swap partition (8G for me). The Ubuntu installer will complain if it’s not there and find it automatically if it is.
    2. Created a Ubuntu 14.04 bootable USB stick: How to create a bootable USB stick on Windows. Install Ubuntu on the ext4 partition.
    3. Created a bootable Windows 7 USB stick. The Universal USB Installer above works fine for this. Install Windows 7 on the Windows partition.
    4. After Step #3 the system will only boot Windows. Use Boot-Repair (option #2) to re-install GRUB.
  • Ubuntu 14.04 seems to work flawlessly on the X1. There were only two hardware compatibility issues that I read about:
    1. Not waking up from suspend mode. This is resolved by updating the BIOS firmware.  Upgrading from v1.09 to v1.15 fixed it for me. The Lenovo firmware only comes as a CD image (.iso) or a Windows update application. Because the X1 does not have a CDROM drive the only reasonable way to upgrade is via Windows. People have upgraded the firmware via USB (see BIOS Upgrade/X Series), but it’s really ugly.
    2. Fingerprint reader. Haven’t tried to use it, and probably won’t.

Happy Ending (I hope)

Like most things in life, nothing is ever perfect. This experience was no exception.

I have a JRuby/Rails project with some Rspec tests that take 80 seconds to complete on the T400 and 20 seconds on the X1. I can live with that improvement. 🙂

Hopefully the X1 will last as long the T400 did.

Brain-Like Chip With 4000 Processor Cores

left-right-brainIBM Unveils a ‘Brain-Like’ Chip With 4,000 Processor Cores. The TrueNorth chip mimics 1 million neurons and 256 million synapses that IBM calls “spiking neurons.”

…the chip can encode data as patterns of pulses, which is similar to one of the many ways neuroscientists think the brain stores information.

IBM Research: Neurosynaptic chips provides more information on the low power system architecture and potential applications:


This is similar to Qualcomm’s Brain-Inspired Computing effort.

Brain-Inspired Computing

zeroth-npuBringing artificial intelligence to mobile computing is a significant challenge. That’s the goal of Qualcomm’s new Zeroth Processors.

Mimicking the human nervous system and brain to allow computers to learn about their environment and modify their behavior based on this information has long been the goal of artificial neural networks.  Whatever computing model is used to achieve this capability the real problem is one of scale. The human brain is estimated to have 100 billion neurons — with 100 trillion connections. That is at least 1,000 times the number of stars in our galaxy.

These computational models can be implemented in software (e.g. Grok), but the ability to scale to the levels required for even simple human-like interactions is severely limited by conventional computing platforms.  The Zeroth Neural Processing Unit (NPU) is a hardware implementation of the brain’s spiking neural networks (SNN) method of information transmission. Integrating the NPU into computing platforms at the chip level would begin to address the computational and power requirements for these types of applications.

The goals of the Zeroth* platform are:

  1. Biologically Inspired Learning
  2. Enable Devices To See and Perceive the World as Humans Do
  3. Creation and definition of an Neural Processing Unit—NPU

Achieving “human-like interaction and behavior” is an ambitious goal, but it seems like this is a good first step.

UPDATE (25-Oct-13): Good overview here: Chips ‘Inspired’ By The Brain Could Be Computing’s Next Big Thing.

UPDATE (1-Jan-14): CES 2014: Intel launches RealSense brand, aims to interface with your brain in the long run

* The name Zeroth comes from the science fiction Three Laws of Robotics. The First law was that “A robot may not harm a human being.”

Asimov once added a “Zeroth Law”—so named to continue the pattern where lower-numbered laws supersede the higher-numbered laws—stating that a robot must not harm humanity.

We’ll have to wait and see, but let’s hope so!

Guest Article: RFID Systems in Healthcare Institutions

Patient, medication, and equipment asset tracking are critical functions for any healthcare organization.  Yedidia Blonder of Vizbee RFID Solutions, a company providing RFID solutions for healthcare applications and other industries, provides an introduction to RFID technology and its benefits.

What healthcare executive wouldn’t want a system that:

  • Helps nurses locate necessary equipment in seconds?
  • Ensures that only the mother of a newborn or a nurse could remove that baby from the nursery?
  • Makes sure patients don’t wander into staff only areas?
  • Lists inventory of all the medications in a large medicine storage area in minutes?
  • Ensures even equipment distribution across wings and prevents theft?
  • Tracks disinfection patterns of employees?

Enter RFID.

RFID (radio frequency identification) is a technology in which radio waves emitted from electronic tags identify them uniquely. The tags are often used to pinpoint the location of the object, or person, to which the tag is attached. This is different than barcode technology, which is usually used to identify an object as belonging to a larger category without individual identification. Barcodes also need to be read one-by-one from very close proximity, whereas RFID readers can read many tags with a single pass of an RFID reader a few meters away.

How does RFID work?

First you need the tags. RFID tags can be split into two main categories: active tags and passive tags. The active tags are battery operated and transmit their data periodically to readers. Their reading distance varies between a few meters to hundreds. Passive tags are much smaller (sometimes like a paper sticker) and do not transmit their data until being interrogated by a reader in their proximity. The passive tags’ reading distance can reach 2-3 meters. Passive tags are usually used for inventory purposes.

The readers consist of an RFID antenna connected to an RFID reader. They receive the data from the tags and then, in order to have a functioning system which will do all the above tasks, transmit the data to a software system which manages the received data.

When the system receives the data, it will both store it for immediate or later review by the healthcare staff, as well as act according to predefined rules set by the administrator. For example, in the case of preventing equipment theft, a rule could be set that if tags attached to pieces of expensive lab equipment go past the reader stationed near the exit to the lab, its signal will set off an alarm, alert important staff members, and lock the exits to that wing of the hospital.

How can RFID help a healthcare institution?

Keeping in mind the stunning figure of 15% of hospital equipment stolen annually as well as the damage that improper maintenance causes, RFID tracking can significantly diminish losses and increase efficient use of equipment. It can ensure that only the right person uses or moves any given piece of equipment, guarantee the correct quantities of a certain apparatus in a designated zone, enable the immediate and accurate location of any item, indicate which item is in use or available, and the list goes on.

RFID can also provide an accurate and comprehensive picture of the total amount of the organization’s inventory, including expiry dates and amount of usage, and provide real-time data on parameters such as temperature and moisture levels, providing alerts in the case of inappropriate conditions that could damage equipment and medications.

Add to this the capacity to track patient and staff movement and interactions with other people and objects – and your RFID healthcare system gives you your entire hospital at a glance, and alerts you to problems.

Implementing RFID systems.

RFID technology is also getting easier to customize. In the past, often RFID hardware would be programmed to work only with specific software. Recently, there have been advances in RFID technology enabling administrators to choose hardware and software independently according to the unique needs of each project. Parameterization tools built into the software can customize applications to specific projects while enabling the implementation of RFID projects in a very short time (days to weeks). You no longer have the time, expense and risk that come with developing software just for your project.

With RFID systems, managing healthcare institutions is getting easier, safer and more efficient.

UPDATE (9/8/2012) : Tim has written an excellent article on the subject:  RFID RTLS Update – Where to Start

Turning the Mind Into a Joystick

More “mind reading” hyperbole in today’s New York Times Magazine: The Cyborg in Us All.

I’ve talked about EEG-related technology many times in the past. Here are some quotes from the article:

This creates a pulse in his brain that travels through the wires into a computer. Thus, a thought becomes a software command.

We’re close to being able to reconstruct the actual music heard in the brain and play it.

… a “telepathy helmet” that would allow soldiers to beam thoughts to one another.

The NeuralPhone was meant to demonstrate that one day we might mind-control the contact lists on our phones.

The general public has two reactions when the lay press publishes this kind of stuff:

  1. I always knew this would come true. I.e. perpetuation of scientific fantasies.
  2. This is really scary stuff. I don’t want anybody reading my mind — or worse, controlling it.
If you know anything about the underlying techniques and algorithms you also know that “mind reading” and useful brain-controlled interfaces are a long way off.  Because the article fails to provide any sort of time-frame perspective, why won’t someone think these capabilities exist now.
The real problem I have with these kinds of articles is that this is important work that could potentially improve the quality of life for many disabled individuals.  Hyping it up to be something it’s not doesn’t help anyone.
One more quote:

“This is freaky.” And it was.

Huh? … I think the NYT needs to improve their editorial oversight.


The Cardiocam: Physiological Monitoring via Webcam

Today’s New York Times Magazine The Year in Ideas: 10th Anniversary Special features the MIT Cardiocam:

Cardiocam is a low-cost, non-contact technology for measurement of physiological signals using a basic digital imaging device such as a Webcam. The ability to perform remote measurements of vital signs is promising for enhancing the delivery of primary health care.

Medgadget covered this in October: MIT Student Uses Webcam to Measure Heart Rate From a Distance includes a video that shows how the Cardiocam is used to create a “medical mirror” for home health monitoring.

A link to a PDF (here) has a full description of the research, including their Cardiac pulse recovery methodology:

The method uses Blind Source (Signal) Separation (BSS) by Independent Component Analysis (ICA) of the changes in the video signal:

Volumetric changes in the facial blood vessels during the cardiac cycle modify the path length of the incident ambient light such that the subsequent changes in amount of reflected light indicate the timing of cardiovascular events.

Very cool.

The BCI X Prize

As announced at a recent MIT workshop: The BCI X PRIZE: This Time It’s Inner Space:

The Brain-Computer Interface (BCI) X PRIZE will reward nothing less than a team that provides vision to the blind, new bodies to disabled people, and perhaps even a geographical “sixth sense” akin to a GPS iPhone app in the brain.

As I’ve discussed many times (e.g. BCI: Brain Computer Interface), “mind reading” with EEG is a huge challenge. Another hurtle they have to overcome:

The foundation must court donors to make the $10 million+ prize a reality. Once funding is secured,…

That will be the easy part.

The problem with the X Prize incentive approach is one of expectations.  If people believe that Avatar-like advances (“new bodies”) is a realisitic result, they will be sorely disappointed.

Even though I’m a certified “mind reading” skeptic I think great BCI strides will inevitably be made. The good news is that these innovations will provide numerous benefits for handicapped individuals.

UPDATE (2/5/10): Here’s a great example: Technology Behind Second Sight Retinal Prosthesis

Microsoft Research at PDC2008

Most of the press coming out of PDC2008 were all the cool new product development technology announcements. What you probably don’t appreciate is the depth and breadth of the Microsoft research effort that is really the foundation for many of these products.

The Day Three PDC Keynote by Rick Rashid (and others) is over 90 minutes long, but is worth a look.  It’s all fascinating stuff.

Hat tip: Dr. Neil’s Notes: Day Three PDC Keynote: Microsoft Research Magic

Medical Devices in Home Health Care

If a company like Intel gets involved you know that providing home health screening devices must be a big opportunity.

Intel Health Guide is designed to be a comprehensive home monitoring service.

  1. In-home patient device.
  2. An online interface allowing clinicians to monitor patients and remotely manage care.
  3. Interactive tools for personalized care management.
  4. Integrates vital sign collection.
  5. Patient reminders.
  6. Multimedia educational content and feedback.
  7. Communications tools such as video conferencing and e-mail.
  8. Can connect to specific models of wired and wireless medical devices:
  • blood pressure monitors
  • glucose meters
  • pulse oximeters
  • peak flow meters
  • weight scales

The Health Guide stores and displays the collected information on a touch screen and sends to a secure host server, where health care professionals can review the information. Patients using the Health Guide can monitor their health status, communicate with care teams and learn about their medical conditions.

Another recent announcement is by Freescale Semiconductor: Here comes an ECG-on-a-chip solution!. The Freescale Electrocardiogram (ECG) Hardware Solution along with Monebo ECG Monitoring Algorithms will allow for low-cost integration of ECG monitoring for remote monitoring and even in home-based devices.

There are many health conditions that would benefit from improved remote monitoring capabilities, but heart disease is certainly at the top of the list and has been shown to reduce hospital re-admissionsHolter monitoring has been around for a long time, but this type of embedded ECG hardware and software technology along with interactive devices like the Intel Health Guide, could significantly raise the bar for ambulatory patient heart monitoring.  Companies like CardioNet are already counting on this trend.

It’s a good bet that PHR providers like Google Health and Microsoft HealthVault will start to incorporate similar interactive technologies into their offerings. Also, Microsoft offers a device certification program that will draw in more devices. Google has similar developer and branding policies in place (see here).

UPDATE (8/19/08): Tools Help Patients Interface With Doctors which also discusses the Zuri device.

UPDATE (11/10/08): Intel Health Guide Undergoing Trials

UPDATE (12/17/08): Intel Health Guide



Twitter Updates