Efficient large language models: A survey

Z Wan, X Wang, C Liu, S Alam, Y Zheng, J Liu… - arxiv preprint arxiv …, 2023 - arxiv.org
Large Language Models (LLMs) have demonstrated remarkable capabilities in important
tasks such as natural language understanding and language generation, and thus have the …

Adoption of transformer neural network to improve the diagnostic performance of oximetry for obstructive sleep apnea

MA Almarshad, S Al-Ahmadi, MS Islam… - Sensors, 2023 - mdpi.com
Scoring polysomnography for obstructive sleep apnea diagnosis is a laborious, long, and
costly process. Machine learning approaches, such as deep neural networks, can reduce …

Density of states prediction of crystalline materials via prompt-guided multi-modal transformer

N Lee, H Noh, S Kim, D Hyun… - Advances in Neural …, 2024 - proceedings.neurips.cc
The density of states (DOS) is a spectral property of crystalline materials, which provides
fundamental insights into various characteristics of the materials. While previous works …

Fcg-Former: Identification of Functional Groups in FTIR Spectra Using Enhanced Transformer-Based Model

VHM Doan, CD Ly, S Mondal, TT Truong… - Analytical …, 2024 - ACS Publications
Deep learning (DL) is becoming more popular as a useful tool in various scientific domains,
especially in chemistry applications. In the infrared spectroscopy field, where identifying …

Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding

C Zhang, Y Tu, Y Zhao, C Yuan, H Chen… - arxiv preprint arxiv …, 2024 - arxiv.org
Modeling and leveraging layout reading order in visually-rich documents (VrDs) is critical in
document intelligence as it captures the rich structure semantics within documents. Previous …

Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks

M Seraphim, A Lechervy, F Yger, L Brun, O Etard - SN Computer Science, 2024 - Springer
In sleep medicine, assessing the evolution of a subject's sleep often involves the costly
manual scoring of electroencephalographic (EEG) signals. In recent years, a number of …

Document Understanding with Deep Learning Techniques

KAL Nguyen - 2024 - theses.hal.science
The field of Document Understanding, which addresses the problem of solving an array of
Natural Language Processing tasks for visually-rich documents, faces challenges due to the …

On the Role of Attention Maps in Visual Transformers—A Clustering Perspective

E ANTTILA RYDERUP, YUP HSU - 2024 - gupea.ub.gu.se
This thesis delves into a novel area of research, exploring whether attention maps from a
single-layer Vision Transformer model exhibit a clustering structure. The discovery of such a …

[PDF][PDF] POSITIONAL ENCODING FOR TRANSFORMERS

K Antipova, H Horban - Publishing House “Baltija Publishing”, 2024 - baltijapublishing.lv
The attention mechanism is a powerful and effective method utilized in natural language
processing. This mechanism allows the model to focus on important parts of the input …