Transformer-based deep learning for predicting protein properties in the life sciences

A Chandra, L Tünnermann, T Löfstedt, R Gratz - Elife, 2023 - elifesciences.org
Recent developments in deep learning, coupled with an increasing number of sequenced
proteins, have led to a breakthrough in life science applications, in particular in protein …

End-to-end transformer-based models in textual-based NLP

A Rahali, MA Akhloufi - Ai, 2023 - mdpi.com
Transformer architectures are highly expressive because they use self-attention
mechanisms to encode long-range dependencies in the input sequences. In this paper, we …

[HTML][HTML] Artificial intelligence-enabled intelligent assistant for personalized and adaptive learning in higher education

R Sajja, Y Sermet, M Cikmaz, D Cwiertny, I Demir - Information, 2024 - mdpi.com
This paper presents a novel framework, artificial intelligence-enabled intelligent assistant
(AIIA), for personalized and adaptive learning in higher education. The AIIA system …

Direct fit to nature: an evolutionary perspective on biological and artificial neural networks

U Hasson, SA Nastase, A Goldstein - Neuron, 2020 - cell.com
Evolution is a blind fitting process by which organisms become adapted to their
environment. Does the brain use similar brute-force fitting processes to learn how to …

[HTML][HTML] From word embeddings to pre-trained language models: A state-of-the-art walkthrough

M Mars - Applied Sciences, 2022 - mdpi.com
With the recent advances in deep learning, different approaches to improving pre-trained
language models (PLMs) have been proposed. PLMs have advanced state-of-the-art …

Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover

L Fröhling, A Zubiaga - PeerJ Computer Science, 2021 - peerj.com
The recent improvements of language models have drawn much attention to potential cases
of use and abuse of automatically generated text. Great effort is put into the development of …

Vision–language model for visual question answering in medical imagery

Y Bazi, MMA Rahhal, L Bashmal, M Zuair - Bioengineering, 2023 - mdpi.com
In the clinical and healthcare domains, medical images play a critical role. A mature medical
visual question answering system (VQA) can improve diagnosis by answering clinical …

SUPERB-SG: Enhanced speech processing universal performance benchmark for semantic and generative capabilities

HS Tsai, HJ Chang, WC Huang, Z Huang… - arxiv preprint arxiv …, 2022 - arxiv.org
Transfer learning has proven to be crucial in advancing the state of speech and natural
language processing research in recent years. In speech, a model pre-trained by self …

[HTML][HTML] Recent progress in the discovery and design of antimicrobial peptides using traditional machine learning and deep learning

J Yan, J Cai, B Zhang, Y Wang, DF Wong, SWI Siu - Antibiotics, 2022 - mdpi.com
Antimicrobial resistance has become a critical global health problem due to the abuse of
conventional antibiotics and the rise of multi-drug-resistant microbes. Antimicrobial peptides …

Protein language-model embeddings for fast, accurate, and alignment-free protein structure prediction

K Weissenow, M Heinzinger, B Rost - Structure, 2022 - cell.com
Advanced protein structure prediction requires evolutionary information from multiple
sequence alignments (MSAs) from evolutionary couplings that are not always available …