Watch a face saying one syllable while hearing another, and you perceive a third that was never present in either. Speech perception turns out to be visual as well as auditory, and knowing about the illusion does not weaken it.
Harry McGurk and John MacDonald discovered it in 1976 by accident, while preparing stimuli for a study of infant speech perception. A technical error paired the wrong audio with the wrong video, and the result was not what either channel contained.
The standard version dubs the sound of ba onto video of a face articulating ga. Most listeners hear da, which is in neither stream.
The explanation is that da is a compromise. The lips are visibly open, which rules out the lip closure that ba requires, and the auditory signal does not match ga, so the perceptual system settles on a place of articulation between the two.
Close your eyes and you hear ba correctly. Open them and da returns immediately. The effect is not a judgement or an inference, and it does not diminish with knowledge of what is happening. This is what makes it a genuine perceptual illusion rather than a bias in reporting.
Speech perception was long treated as an auditory process with vision as an optional aid, useful in noise. The effect shows that the two are combined before a percept is formed, and that the combination is not under voluntary control.
The contribution of vision is substantial and continuous. Seeing a speaker's face improves intelligibility in noise by an amount equivalent to a large improvement in signal-to-noise ratio, which is why conversation in a loud room is much easier face to face.
The effect is strongest when the auditory signal is ambiguous or degraded, which fits an account in which the brain weights each source of evidence by its reliability. That framework, treating perception as combining uncertain evidence in proportion to its precision, describes the data well and has been applied across the senses.
The illusion is not uniform, and the variation has been informative.
It is weaker in children than adults, which suggests the integration is learned rather than innate, or at least that it strengthens with experience of speech.
It varies by language. Japanese speakers show weaker effects than English speakers in several studies, which has been linked to cultural differences in looking at faces during conversation, though the finding is debated.
It is reduced in some autistic individuals, and in people with certain lesions of the superior temporal sulcus, which is the region most consistently implicated in audiovisual integration.
Individual variation among typical adults is large and stable. Some people show the illusion on nearly every trial and others rarely, and the difference persists on retesting.
The effect is robust and its existence is not in question. Its interpretation has narrowed since the 1970s.
It was taken for a time as strong support for the motor theory of speech perception, which holds that speech is perceived by recovering the articulatory gestures that produced it. That theory has other difficulties and is no longer widely held, and the effect is compatible with accounts that do not require it.
The large individual variation also complicates its use as a general measure. Studies using it as a proxy for multisensory integration ability have produced inconsistent results, and a 2018 analysis argued that its reliability as an individual difference measure is lower than commonly assumed.
Practical consequences follow directly. Poorly synchronised audio and video in broadcasting and video calls degrades intelligibility, not merely aesthetics. Dubbed film is harder to follow than subtitled film for reasons that are perceptual rather than a matter of taste. Face coverings reduce speech intelligibility by more than their acoustic attenuation alone accounts for, because they remove the visual channel, which matters particularly for people with hearing loss.
The broader lesson is the one the effect is usually cited for. Perception is not a recording of the senses but a construction from them, and the construction can be shown to produce something that no sense delivered.
