Multimodal deep learning systems that employ multiple modalities like text, image, audio,
video, etc., are showing better performance than individual modalities (ie, unimodal) …

In remote sensing (RS), multiple modalities of data are usually available, eg, RGB,
multispectral, hyperspectral, light detection and ranging (LiDAR), and synthetic aperture …

Building upon the considerable advances in Large Language Models (LLMs), we are now
equipped to address more sophisticated tasks demanding a nuanced understanding of …

The concept of augmented reality (AR) assistants has captured the human imagination for
decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary to …

With the development of multimodal systems and natural language generation techniques,
the resurgence of multimodal datasets has attracted significant research interests, which …

Recipe texts are an idiosyncratic form of instructional language that pose unique challenges
for automatic understanding. One challenge is that a cooking step in one recipe can be …

Can we teach a robot to recognize and make predictions for activities that it has never seen
before? We tackle this problem by learning models for video from text. This paper presents a …

Understanding procedural natural language (eg, step-by-step instructions) is a crucial step
to execution and planning. However, while there are ample corpora and downstream tasks …

Procedural Multimodal Documents (PMDs) organize textual instructions and corresponding
images step by step. Comprehending PMDs and inducing their representations for the …

This tutorial provides a comprehensive and in-depth view of the research on procedures,
primarily in Natural Language Processing. A procedure is a sequence of steps intended to …