When politicians address a public audience, they often emphasize the most important words in their speeches by accompanying them with rhythmic gestures, such as moving their hands up and down, for example.
In fact, it seems that we all do this unconsciously, as “hand claps” are among the most common gestures in everyday conversations. People notably use these gestures with great precision alongside prominent words and important points during discussions. But do rhythmic gestures actually help listeners understand the speaker? Researchers Hans Rutger Bockstael from the Max Planck Institute for Psycholinguistics and Professor David Peters from Tilburg University examined whether what we see really shapes what we hear. Bockstael states, "In face-to-face communication, language becomes much more than just speech, as speakers use various channels including the mouth, hands, and face to convey a message. We wanted to understand how listeners benefit from these different streams of information when listening to someone."
It is worth mentioning that in a known illusion called the "McGurk effect," people hear sounds of certain letters, such as "b" in the word "ba," as different sounds, as if they were "pa" or "fa," depending on the lip movements of the speakers. We can identify the sound simply by observing lip movements. During the study, researchers selected a set of Dutch words that differed only in their stress patterns. For instance, the word "PLAto" with stress on the first syllable, referring to the ancient Greek philosopher Plato, is different from "plaTO" with stress on the second syllable, which means a plateau.
In the study, participants watched a video of Bockstael producing words while performing various rhythmic gestures such as, “Now I say the word... Plato,” and then participants had to identify which word they heard (PLAto or plaTEAU) to determine whether rhythmic gestures indicating stress at the beginning or the end of the word could affect their understanding. According to the researchers, listeners were more likely to hear stress on a syllable if there was a rhythmic gesture accompanying that syllable. The most surprising finding was that rhythmic gestures affected what people heard in vowel sounds, as the length of a vowel is typically associated with the stress pattern in the word.
Bockstael states, “Listeners don’t just listen with their ears, but also with their eyes, and these results are the first to show that rhythmic gestures affect the sounds of speech that we hear.” Notably, both Bockstael and Peters believe that the impact of rhythmic gestures might be greater in real life, especially when speech is less clear than it was in the study. In noisy listening conditions, visible rhythmic gestures may be the most important for successful communication, with Bockstael jokingly adding, “Wash your hands well and use them.”
