Recent advances and applications of deep learning methods in materials science

K Choudhary, B DeCost, C Chen, A Jain… - npj Computational …, 2022 - nature.com
Deep learning (DL) is one of the fastest-growing topics in materials data science, with
rapidly emerging applications spanning atomistic, image-based, spectral, and textual data …

Pre-trained models for natural language processing: A survey

X Qiu, T Sun, Y Xu, Y Shao, N Dai, X Huang - Science China …, 2020 - Springer
Recently, the emergence of pre-trained models (PTMs) has brought natural language
processing (NLP) to a new era. In this survey, we provide a comprehensive review of PTMs …

Explainability in graph neural networks: A taxonomic survey

H Yuan, H Yu, S Gui, S Ji - IEEE transactions on pattern …, 2022 - ieeexplore.ieee.org
Deep learning methods are achieving ever-increasing performance on many artificial
intelligence tasks. A major limitation of deep models is that they are not amenable to …

On explainability of graph neural networks via subgraph explorations

H Yuan, H Yu, J Wang, K Li, S Ji - … conference on machine …, 2021 - proceedings.mlr.press
We consider the problem of explaining the predictions of graph neural networks (GNNs),
which otherwise are considered as black boxes. Existing methods invariably focus on …

Don't stop pretraining: Adapt language models to domains and tasks

S Gururangan, A Marasović, S Swayamdipta… - arxiv preprint arxiv …, 2020 - arxiv.org
Language models pretrained on text from a wide variety of sources form the foundation of
today's NLP. In light of the success of these broad-coverage models, we investigate whether …

Language models with image descriptors are strong few-shot video-language learners

Z Wang, M Li, R Xu, L Zhou, J Lei… - Advances in …, 2022 - proceedings.neurips.cc
The goal of this work is to build flexible video-language models that can generalize to
various video-to-text tasks from few examples. Existing few-shot video-language learners …

[HTML][HTML] Exploring an AI-based writing Assistant's impact on English language learners

JM Gayed, MKJ Carlon, AM Oriola, JS Cross - Computers and Education …, 2022 - Elsevier
The increasing use of English as a Lingua Franca (ELF) worldwide has brought attention to
tools that can assist English as a Foreign Language (EFL) learners in their journey to …

Relational graph attention network for aspect-based sentiment analysis

K Wang, W Shen, Y Yang, X Quan, R Wang - arxiv preprint arxiv …, 2020 - arxiv.org
Aspect-based sentiment analysis aims to determine the sentiment polarity towards a specific
aspect in online reviews. Most recent efforts adopt attention-based neural network models to …

Declutr: Deep contrastive learning for unsupervised textual representations

J Giorgi, O Nitski, B Wang, G Bader - arxiv preprint arxiv:2006.03659, 2020 - arxiv.org
Sentence embeddings are an important component of many natural language processing
(NLP) systems. Like word embeddings, sentence embeddings are typically learned on large …

Learning causally invariant representations for out-of-distribution generalization on graphs

Y Chen, Y Zhang, Y Bian, H Yang… - Advances in …, 2022 - proceedings.neurips.cc
Despite recent success in using the invariance principle for out-of-distribution (OOD)
generalization on Euclidean data (eg, images), studies on graph data are still limited …