Attention is an increasingly popular mechanism used in a wide range of neural
architectures. The mechanism itself has been realized in a variety of formats. However …

The field of artificial intelligence has experienced a dramatic methodological shift towards
large neural networks trained on plentiful data. This shift has been fueled by recent …

Large pre-trained neural networks such as BERT have had great recent success in NLP,
motivating a growing body of research investigating what aspects of language they are able …

Human language is often multimodal, which comprehends a mixture of natural language,
facial gestures, and acoustic behaviors. However, two major challenges in modeling such …

Multi-headed attention is a driving force behind recent state-of-the-art NLP models. By
applying multiple attention mechanisms in parallel, it can express sophisticated functions …

In humans, Attention is a core property of all perceptual and cognitive operations. Given our
limited ability to process competing sources, attention mechanisms select, modulate, and …

Natural Language Processing (NLP) helps empower intelligent machines by enhancing a
better understanding of the human language for linguistic-based human-computer …

Contextualized representation models such as ELMo (Peters et al., 2018a) and BERT
(Devlin et al., 2018) have recently achieved state-of-the-art results on a diverse array of …

We present simple BERT-based models for relation extraction and semantic role labeling. In
recent years, state-of-the-art performance has been achieved using neural models by …

Neural extractive summarization models usually employ a hierarchical encoder for
document encoding and they are trained using sentence-level labels, which are created …