[HTML][HTML] A survey of transformers

T Lin, Y Wang, X Liu, X Qiu - AI open, 2022 - Elsevier
Transformers have achieved great success in many artificial intelligence fields, such as
natural language processing, computer vision, and audio processing. Therefore, it is natural …

Neural machine translation for low-resource languages: A survey

S Ranathunga, ESA Lee, M Prifti Skenduli… - ACM Computing …, 2023 - dl.acm.org
Neural Machine Translation (NMT) has seen tremendous growth in the last ten years since
the early 2000s and has already entered a mature phase. While considered the most widely …

[PDF][PDF] A survey of large language models

WX Zhao, K Zhou, J Li, T Tang… - arxiv preprint arxiv …, 2023 - paper-notes.zhjwpku.com
Ever since the Turing Test was proposed in the 1950s, humans have explored the mastering
of language intelligence by machine. Language is essentially a complex, intricate system of …

Chatgpt or human? detect and explain. explaining decisions of machine learning model for detecting short chatgpt-generated text

S Mitrović, D Andreoletti, O Ayoub - arxiv preprint arxiv:2301.13852, 2023 - arxiv.org
ChatGPT has the ability to generate grammatically flawless and seemingly-human replies to
different types of questions from various domains. The number of its users and of its …

On the explainability of natural language processing deep models

JE Zini, M Awad - ACM Computing Surveys, 2022 - dl.acm.org
Despite their success, deep networks are used as black-box models with outputs that are not
easily explainable during the learning and the prediction phases. This lack of interpretability …

Transformers: State-of-the-art natural language processing

T Wolf, L Debut, V Sanh, J Chaumond… - Proceedings of the …, 2020 - aclanthology.org
Recent progress in natural language processing has been driven by advances in both
model architecture and model pretraining. Transformer architectures have facilitated …

Huggingface's transformers: State-of-the-art natural language processing

T Wolf, L Debut, V Sanh, J Chaumond… - arxiv preprint arxiv …, 2019 - arxiv.org
Recent progress in natural language processing has been driven by advances in both
model architecture and model pretraining. Transformer architectures have facilitated …

On layer normalization in the transformer architecture

R **ong, Y Yang, D He, K Zheng… - International …, 2020 - proceedings.mlr.press
The Transformer is widely used in natural language processing tasks. To train a Transformer
however, one usually needs a carefully designed learning rate warm-up stage, which is …

Scaling up models and data with t5x and seqio

A Roberts, HW Chung, G Mishra, A Levskaya… - Journal of Machine …, 2023 - jmlr.org
Scaling up training datasets and model parameters have benefited neural network-based
language models, but also present challenges like distributed compute, input data …

A comparative study on transformer vs rnn in speech applications

S Karita, N Chen, T Hayashi, T Hori… - 2019 IEEE automatic …, 2019 - ieeexplore.ieee.org
Sequence-to-sequence models have been widely used in end-to-end speech processing,
for example, automatic speech recognition (ASR), speech translation (ST), and text-to …