This finger-mounted camera turns any book into an audiobook

 

 

Camera technology goes far beyond taking beautiful images. One such example is FingerReader, a finger-mounted device developed by researchers at MIT’s Media Lab that enables the blind to read without the need for Braille.

Currently a prototype, the device relies on a small camera, dedicated sensors, character recognition software and a set of algorithms to read aloud words that are printed on a page or displayed on a digital reading device, such as a Kindle.

In speaking to CNET, Roy Shilkrot, MIT graduate student and co-lead author of the prototype’s accompanying research said:

‘You really need to have a tight coupling between what the person hears and where the fingertip is […] They really need a fast, real-time feedback to maintain this connection. If it’s broken, it breaks the illusion [of seamless translation].’

As shown off in the video, below, the device automatically calculates where one line ends and the next begins, based on the typographic nature of the words. After determining where the words are, the camera is used to capture individual snippets, which are sent through the software that turns the digital words into audible sounds.

 

A look at what the software sees and parses through as a reader's finger glides across the page.

All of this is being done in real-time, remember. Currently, FingerReadder needs to be tethered to a laptop, which handles the heavy computing. But, the team is working hard to develop an open-source version of the software that will be able to use Android phones as the ‘brain’ of the device.

CNET notes the team is working to develop haptic and audio feedback to help guide the readers’ fingers along the lines in case they start to drift. The haptic engine would rely on two motors: one on top and one on the bottom of the device to ‘bump’ readers' finger up and down, which would be accompanied by an audible tone.

We often constrain camera technology to only visual art, but for some visual art isn’t an option. In such cases, it’s important to think outside the box; to use cameras and technology together to bring them art and beauty in other forms.

Image credits: Images courtesy of MIT Media Labs