Inspired by the human cognitive system, attention is a mechanism that imitates the human
cognitive awareness about specific information, amplifying critical details to focus more on …

In humans, Attention is a core property of all perceptual and cognitive operations. Given our
limited ability to process competing sources, attention mechanisms select, modulate, and …

The recently proposed BERT has shown great power on a variety of natural language
understanding tasks, such as text classification, reading comprehension, etc. However, how …

Deep learning technology provides a promising approach for rotary machine fault diagnosis
(RMFD), where vibration signals are commonly utilized as input of a deep network model to …

There is a recent surge of interest in using attention as explanation of model predictions,
with mixed evidence on whether attention can be used as such. While attention conveniently …

This paper studies semi-supervised object classification in relational data, which is a
fundamental problem in relational data modeling. The problem has been extensively studied …

Normalization layers are a staple in state-of-the-art deep neural network architectures. They
are widely believed to stabilize training, enable higher learning rate, accelerate …

Attention mechanisms have become ubiquitous in NLP. Recent architectures, notably the
Transformer, learn powerful context-aware word representations through layered, multi …

Knowledge-grounded dialogue is a task of generating an informative response based on
both discourse context and external knowledge. As we focus on better modeling the …

Self-attention based Transformer has demonstrated the state-of-the-art performances in a
number of natural language processing tasks. Self-attention is able to model long-term …