Attention mechanism in neural networks: where it comes and where it goes
D Soydaner - Neural Computing and Applications, 2022 - Springer
A long time ago in the machine learning literature, the idea of incorporating a mechanism
inspired by the human visual system into neural networks was introduced. This idea is …
inspired by the human visual system into neural networks was introduced. This idea is …
Attention, please! A survey of neural attention models in deep learning
In humans, Attention is a core property of all perceptual and cognitive operations. Given our
limited ability to process competing sources, attention mechanisms select, modulate, and …
limited ability to process competing sources, attention mechanisms select, modulate, and …
Normalized and geometry-aware self-attention network for image captioning
Self-attention (SA) network has shown profound value in image captioning. In this paper, we
improve SA from two aspects to promote the performance of image captioning. First, we …
improve SA from two aspects to promote the performance of image captioning. First, we …
RKT: relation-aware self-attention for knowledge tracing
S Pandey, J Srivastava - Proceedings of the 29th ACM international …, 2020 - dl.acm.org
The world has transitioned into a new phase of online learning in response to the recent
Covid19 pandemic. Now more than ever, it has become paramount to push the limits of …
Covid19 pandemic. Now more than ever, it has become paramount to push the limits of …
CAAN: Context-aware attention network for visual question answering
C Chen, D Han, CC Chang - Pattern Recognition, 2022 - Elsevier
Understanding multimodal information is the key to visual question answering (VQA) tasks.
Most existing approaches use attention mechanisms to acquire fine-grained information …
Most existing approaches use attention mechanisms to acquire fine-grained information …
Atloc: Attention guided camera localization
Deep learning has achieved impressive results in camera localization, but current single-
image techniques typically suffer from a lack of robustness, leading to large outliers. To …
image techniques typically suffer from a lack of robustness, leading to large outliers. To …
Context-guided bert for targeted aspect-based sentiment analysis
Aspect-based sentiment analysis (ABSA) and Targeted ASBA (TABSA) allow finer-grained
inferences about sentiment to be drawn from the same text, depending on context. For …
inferences about sentiment to be drawn from the same text, depending on context. For …
Convolutional self-attention networks
Self-attention networks (SANs) have drawn increasing interest due to their high
parallelization in computation and flexibility in modeling dependencies. SANs can be further …
parallelization in computation and flexibility in modeling dependencies. SANs can be further …
Modeling intra-relation in math word problems with different functional multi-head attentions
Several deep learning models have been proposed for solving math word problems (MWPs)
automatically. Although these models have the ability to capture features without manual …
automatically. Although these models have the ability to capture features without manual …
Spatial-temporal convolutional graph attention networks for citywide traffic flow forecasting
Traffic flow prediction plays an important role in many spatial-temporal data applications, eg,
traffic management and urban planning. Various deep learning techniques are developed to …
traffic management and urban planning. Various deep learning techniques are developed to …