Efficient large language models: A survey
Large Language Models (LLMs) have demonstrated remarkable capabilities in important
tasks such as natural language understanding and language generation, and thus have the …
tasks such as natural language understanding and language generation, and thus have the …
Adoption of transformer neural network to improve the diagnostic performance of oximetry for obstructive sleep apnea
Scoring polysomnography for obstructive sleep apnea diagnosis is a laborious, long, and
costly process. Machine learning approaches, such as deep neural networks, can reduce …
costly process. Machine learning approaches, such as deep neural networks, can reduce …
Density of states prediction of crystalline materials via prompt-guided multi-modal transformer
The density of states (DOS) is a spectral property of crystalline materials, which provides
fundamental insights into various characteristics of the materials. While previous works …
fundamental insights into various characteristics of the materials. While previous works …
Fcg-Former: Identification of Functional Groups in FTIR Spectra Using Enhanced Transformer-Based Model
Deep learning (DL) is becoming more popular as a useful tool in various scientific domains,
especially in chemistry applications. In the infrared spectroscopy field, where identifying …
especially in chemistry applications. In the infrared spectroscopy field, where identifying …
Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding
Modeling and leveraging layout reading order in visually-rich documents (VrDs) is critical in
document intelligence as it captures the rich structure semantics within documents. Previous …
document intelligence as it captures the rich structure semantics within documents. Previous …
Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks
In sleep medicine, assessing the evolution of a subject's sleep often involves the costly
manual scoring of electroencephalographic (EEG) signals. In recent years, a number of …
manual scoring of electroencephalographic (EEG) signals. In recent years, a number of …
Document Understanding with Deep Learning Techniques
KAL Nguyen - 2024 - theses.hal.science
The field of Document Understanding, which addresses the problem of solving an array of
Natural Language Processing tasks for visually-rich documents, faces challenges due to the …
Natural Language Processing tasks for visually-rich documents, faces challenges due to the …
On the Role of Attention Maps in Visual Transformers—A Clustering Perspective
E ANTTILA RYDERUP, YUP HSU - 2024 - gupea.ub.gu.se
This thesis delves into a novel area of research, exploring whether attention maps from a
single-layer Vision Transformer model exhibit a clustering structure. The discovery of such a …
single-layer Vision Transformer model exhibit a clustering structure. The discovery of such a …
[PDF][PDF] POSITIONAL ENCODING FOR TRANSFORMERS
K Antipova, H Horban - Publishing House “Baltija Publishing”, 2024 - baltijapublishing.lv
The attention mechanism is a powerful and effective method utilized in natural language
processing. This mechanism allows the model to focus on important parts of the input …
processing. This mechanism allows the model to focus on important parts of the input …