For about a decade, scientists who study speech and hearing have been able to demonstrate that once a person becomes used to the sound of another's voice, that voice becomes easier to understand. But there's been some question about what role vision plays in the process of hearing.
Professor Larry Rosenblum from the University of California at Riverside wanted to study how seeing - and knowing - a speaker's face can influence hearing.
He took 60 people with no formal training in lip reading and had them watch silent videos of someone speaking, so they could get familiar with the face of the speaker. Then the subjects watched videos, with sound, of the same speaker. Half the subjects heard the actual voice of the person in the video, half heard the voice of another person.
Rosenblum says he found a marked difference. "Subjects that first lip read and then heard speech from the same speaker actually were able to hear the speech better than subjects that lip read from one speaker and then heard the speech of a different speaker." He says this indicates that familiarity with a speaker in one sense - vision - can affect another sense - hearing, and speech perception - and that can boost understanding of what's being heard.
Listeners collect extra information from the movements of a speaker's face and body, something Rosenblum calls the idiolect. "So even if two people have the same dialect," he explains, "they are distinguished on their specific way of stringing together vowels and consonants. And that idiolectical information, that speaking style information, is the same information whether it's heard or seen."
Rosenblum says if you can see the person you're talking to, you're probably lip reading on some level. He says this is particularly true in situations where audio input is complicated, say, at a noisy party or listening in a second language. "If you're talking to someone who has a thick foreign accent, it helps to look at the face of the speaker. If the person doesn't have an accent but they're talking about something materially complicated, it helps to see the person that's talking. So it pays to sit up front in a complicated lecture so that you can see the face." He adds that if you're speaking in a language that's not your first language, it helps to see the face of the speaker as well.
Rosenblum says knowing about how we 'hear' speech helps scientists understand more about how the brain processes that speech. He says this information could eventually be useful for people who lose the ability to speak or hear because of brain injuries or strokes. Rosenblum's research will be published in the May issue of the journal Psychological Science.