Fusing eye movements and observer narratives for expert-driven image-region annotations

P Vaidyanathan, E Prud'hommeaux, JB Pelz… - Proceedings of the …, 2016 - dl.acm.org
Human image understanding is reflected by individuals' visual and linguistic behaviors, but
the meaningful computational integration and interpretation of their multimodal …

[PDF][PDF] Using co-captured face, gaze, and verbal reactions to images of varying emotional content for analysis and semantic alignment

A Gangji, T Walden, P Vaidyanathan… - The AAAI-17 …, 2017 - cdn.aaai.org
Analyzing different modalities of expression can provide insights into the ways that humans
interpret, label, and react to images. Such insights have the potential not only to advance our …

Computational framework for fusing eye movements and spoken narratives for image annotation

P Vaidyanathan, E Prud'hommeaux, CO Alm… - Journal of …, 2020 - jov.arvojournals.org
Despite many recent advances in the field of computer vision, there remains a disconnect
between how computers process images and how humans understand them. To begin to …

[КНИГА][B] Visual-Linguistic Semantic Alignment: Fusing Human Gaze and Spoken Narratives for Image Region Annotation

P Vaidyanathan - 2017 - search.proquest.com
Advanced image-based application systems such as image retrieval and visual question
answering depend heavily on semantic image region annotation. However, improvements in …