This Smartphone App Aims to Help the Blind See

Your smartphone will never totally replace a fellow human being. But a new app seeks to function a bit like another set of eyes in the house, helping the visually impaired identify everyday items as they come into contact with them. 

BlindTool is an Android app that can recognize common objects and relay that information to the user through voice, and as a simple text-based message across the screen (for anyone who feels like trying it out). It was created by Joseph Paul Cohen, a Ph.D. candidate at the University of Massachusetts, Boston, who was inspired to create a tool for the visually impaired after working with a blind programmer.

BlindTool works through training and logic. While developing the app, Cohen used a database of around 1000 images to teach the neural network to identify objects like bananas, frying pans, and cups. Using that information, the app can make a guess about whether what's in front of the camera matches a object it already knows, vibrating when it's identified something. The application will only speak the name of an object when it's 30 percent sure of what it is, and the vibration becomes stronger around 90 percent.

The concept seems promising, but BlindTool is not without its quirks (it should be noted that the program is still in its beta stage). A large database with tens of thousands of images would have made the application slow and too large for a smartphone, so Cohen went for speed and efficiency over accuracy. He told FastCoDesign that the photos used to train the app were selected randomly, not geared toward what a user might need to identify on a day-to-day basis.

In a quick test of BlindTool around the mental_floss office, the identifications were hit-and-miss. Objects like keyboards and sunglasses were identified correctly the first time around, but others, like a digital watch and running shoes, took several passes. A toothbrush was never positively identified, and the app was confident that a wastebasket was a toilet seat.

Screenshots via BlindTool

Despite the kinks, BlindTool shows how technology like this may someday be harnessed to make everyday life for the visually impaired easier. It joins a handful of other apps designed to aid the blind, and while it might not be perfect, Cohen told FastCoDesign that he believes it still serves an important purpose: "If someone just wants to look around, and get the gist of what something looks like so they can add a whole other dimension of sight, experience of the world, maybe [BlindTool is technically] wrong, but ... I still think it gives a sense of independence, which is a big goal of doing this."

[h/t: FastCoDesign]