Smart mirrors have been things the retail industry have been toying with for sometime.  The theory is that they’ll allow us to stand in front of a reflective surface and get all manner of interesting things to look at, whether it’s various outfits, different colors and so on.

Not to downplay the potential importance of such inventions, perhaps a more worthwhile version of the smart mirror is being developed by researchers at Cornell Tech, whose mirror is able to scan our skin for changes that the team believe are indicative of illnesses such as cancer.

“We wanted to find something that would benefit your life, that you could do in three seconds every morning,” the team say. “An easy target was skin cancer. It affects one in five Americans over the course of their lifetime, and you can tell a lot about a malignant mole or anomaly just by visual inspection. We thought we could put computer vision and machine learning technology to use to tackle this disease.”

Reflective health

The technology, called Reflective Health, consists of a two-way mirror with a monitor behind it.  The reflective ratio of the mirror allows the user to easily see what’s going on behind the mirror.  The technology will then photograph the user’s face and upper body every day, with a historical profile then created for each mole.  The team hope to eventually provide doctors with a dashboard that can visually display any significant changes.

Suffice to say, there are significant privacy concerns with technology of this nature, and the team are working hard to counter any potential issues.  To overcome these concerns, the team have developed the system specifically to protect the identity of the user.  No internet connection will be required to use the system, with original images destroyed after use, with just the cropped close ups of specific moles retained and shared with both the user and their doctor.

The team are now working on ensuring that the machine learning algorithm is as tight as possible.  They’ve already built a couple of hardware prototypes, so are making good progress.

“We use a simple computer vision technique called a blob detector, which, as it sounds, detects blobs, sections of discoloration,” they explain. “But there’s a lot of noise—hair, stuff on your shirt, etcetera. So we take all of those blobs and essentially take little crops and run them through a machine learning model that’s meant to filter the images to just moles. That’s the part we’re putting a lot of work into now.”

As is common with many machine learning based projects, the key is to ensure they have good data to train the algorithm with.  That hasn’t been easy, but they managed to obtain data from another study that had been conducted by Oregon Health and Science University.

Whilst they’re still some way from offering the technology on the marketplace, it’s an interesting example of where the technology is heading.  It will be especially powerful if it can incorporate numerous conditions, but having an all-inclusive product like that has numerous hurdles to overcome.  Nonetheless, it’s a fascinating technology that will be worth keeping an eye on.