Seeing 
With Sound

Thursday, February 21, 2013

Recent work by French and Israeli researchers revealed that congenitally blind individuals can activate the visual word form area (VWFA) — the portion of the brain used for reading — with the aid of specialized devices that convert visual information into sounds.

As reported in a November 2012 issue of the journal Neuron, Amir Amedi, Ph.D., associate professor of medical neurobiology at the Institute for Medical Research Israel-Canada and the Edmond and Lily Safra Center for Brain Sciences at the Hebrew University of Jerusalem, and team used advanced sensory substitution devices (SSDs) to create “soundscapes,” or information that blind people can “see” using their auditory and other senses.

To create soundscapes, visual-to-auditory SSDs captured images with a camera and converted them into auditory information using an algorithm. Blind SSD users learned to read letters and words and to distinguish among images of faces, houses and other objects.
Using functional magnetic resonance imaging, the team revealed that, as in sighted people, the VWFAs of blind study participants became most active when the individuals read letters — a finding that surprised Dr. Amedi and his team because it was assumed adults who had been blind from birth would be unable to rouse the visual cortex and VWFA so much later in life than when conventional sight normally develops. Most blind study participants experienced heightened VWFA activation due to reading letters after 10 hours of SSD training. One individual, however, experienced increased VWFA activation after only two hours of using the SSD.

Fascinating Insights

“Most people — and 99% of neuroscience textbooks — think about the brain as a sensory machine,” says Dr. Amedi. “We have the visual occipital cortex for vision, the auditory temporal cortex for audition, the somatosensory parietal cortex for touch. However, the fact that the visual cortex responded as strongly to sound input — not only responded, but also showed very similar ‘visual’ specificity to sounds — suggests this is not the case.

“Thus, we suggest the brain is a flexible task machine that if given the proper input and training can, in a matter of hours, change its specificity to process other senses. In a way, it is almost an illusion that this is a visual cortex. With the proper technology and training, the brain can process and respond to any other sense just the same as it does vision. We must keep working to improve SSDs so we can make the next leap, which is to make SSDs useful, affordable and accessible to every blind individual.”

Dr. Amedi plans to continue his work with SSDs by incorporating algorithms for color, improving the pleasantness of users’ experiences and enabling the blind to download device lessons for at-home practice. He also plans to use SSDs to investigate the brain’s plasticity — work that may one day contribute to making lifelong blindness a thing of the past.