Multimodal language processing in human communication

J Holler, SC Levinson - Trends in Cognitive Sciences, 2019 - cell.com
The natural ecology of human language is face-to-face interaction comprising the exchange
of a plethora of multimodal signals. Trying to understand the psycholinguistic processing of …

Perspective on unconventional computing using magnetic skyrmions

O Lee, R Msiska, MA Brems, M Kläui… - Applied Physics …, 2023 - pubs.aip.org
Learning and pattern recognition inevitably requires memory of previous events, a feature
that conventional CMOS hardware needs to artificially simulate. Dynamical systems …

Forty years after hearing lips and seeing voices: the McGurk effect revisited

A Alsius, M Paré, KG Munhall - Multisensory Research, 2018 - brill.com
Since its discovery 40 years ago, the McGurk illusion has been usually cited as a
prototypical paradigmatic case of multisensory binding in humans, and has been extensively …

[HTML][HTML] Audiovisual speech perception: Moving beyond McGurk

KJ Van Engen, A Dey, MS Sommers… - The journal of the …, 2022 - pubs.aip.org
Although it is clear that sighted listeners use both auditory and visual cues during speech
perception, the manner in which multisensory information is combined is a matter of debate …

Web-based live speech-driven lip-sync

G Llorach, A Evans, J Blat, G Grimm… - 2016 8th International …, 2016 - ieeexplore.ieee.org
Virtual characters are an integral part of many games and virtual worlds. The ability to
accurately synchronize lip movement to audio speech is an important aspect in the …

Hand gestures have predictive potential during conversation: An investigation of the timing of gestures in relation to speech

M Ter Bekke, L Drijvers, J Holler - Cognitive Science, 2024 - Wiley Online Library
During face‐to‐face conversation, transitions between speaker turns are incredibly fast.
These fast turn exchanges seem to involve next speakers predicting upcoming semantic …

Audiovisual Synchrony in Left-hemisphere Brain-lesioned Individuals with Aphasia

H Farahbod, C Rogalsky, LM Keator, J Cai… - Journal of Cognitive …, 2025 - direct.mit.edu
We investigated the ability of 40 left-hemisphere brain-lesioned individuals with various
diagnoses of aphasia to temporally synchronize the audio of a spoken word to its congruent …

[HTML][HTML] The effect of eye disease, cataract surgery and hearing aid use on multisensory integration in ageing

RJ Hirst, A Setti, C De Looze, KO Akuffo, T Peto… - cortex, 2020 - Elsevier
Sensory impairment is common in ageing, as are approaches to treat it. However, the impact
of age-related sensory impairment upon multisensory perception remains unexplored …

Auditory and visual distractors disrupt multisensory temporal acuity in the crossmodal temporal order judgment task

CL Dean, BA Eggleston, KD Gibney, E Aligbe… - PLoS …, 2017 - journals.plos.org
The ability to synthesize information across multiple senses is known as multisensory
integration and is essential to our understanding of the world around us. Sensory stimuli that …

The impact and status of Carol Fowler's supramodal theory of multisensory speech perception

LD Rosenblum, J Dorsi, JW Dias - Ecological Psychology, 2016 - Taylor & Francis
One important contribution of Carol Fowler's direct approach to speech perception is its
account of multisensory perception. This supramodal account proposes a speech function …