Does a neural network really encode symbolic concepts?

M Li, Q Zhang - International conference on machine …, 2023 - proceedings.mlr.press
Recently, a series of studies have tried to extract interactions between input variables
modeled by a DNN and define such interactions as concepts encoded by the DNN …

Towards the difficulty for a deep neural network to learn concepts of different complexities

D Liu, H Deng, X Cheng, Q Ren… - Advances in Neural …, 2023 - proceedings.neurips.cc
This paper theoretically explains the intuition that simple concepts are more likely to be
learned by deep neural networks (DNNs) than complex concepts. In fact, recent studies …

Defining and quantifying the emergence of sparse concepts in dnns

J Ren, M Li, Q Chen, H Deng… - Proceedings of the …, 2023 - openaccess.thecvf.com
This paper aims to illustrate the concept-emerging phenomenon in a trained DNN.
Specifically, we find that the inference score of a DNN can be disentangled into the effects of …

Attention-SA: Exploiting Model-approximated Data Semantics for Adversarial Attack

Q Li, Q Hu, H Fan, C Lin, C Shen… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Adversarial Defense of deep neural networks have gained significant attention and there
have been active research efforts on model vulnerabilities for attacking such as gradient …

Data poisoning attacks against conformal prediction

Y Li, A Chen, W Qian, C Zhao, D Lidder… - Forty-first International …, 2024 - openreview.net
The efficient and theoretically sound uncertainty quantification is crucial for building trust in
deep learning models. This has spurred a growing interest in conformal prediction (CP), a …

Towards the dynamics of a DNN learning symbolic interactions

Q Ren, J Zhang, Y Xu, Y **n, D Liu, Q Zhang - arxiv preprint arxiv …, 2024 - arxiv.org
This study proves the two-phase dynamics of a deep neural network (DNN) learning
interactions. Despite the long disappointing view of the faithfulness of post-hoc explanation …

Can we faithfully represent masked states to compute shapley values on a dnn?

J Ren, Z Zhou, Q Chen, Q Zhang - arxiv preprint arxiv:2105.10719, 2021 - arxiv.org
Masking some input variables of a deep neural network (DNN) and computing output
changes on the masked input sample represent a typical way to compute attributions of input …

Where we have arrived in proving the emergence of sparse symbolic concepts in ai models

Q Ren, J Gao, W Shen, Q Zhang - arxiv preprint arxiv:2305.01939, 2023 - arxiv.org
This study aims to prove the emergence of symbolic concepts (or more precisely, sparse
primitive inference patterns) in well-trained deep neural networks (DNNs). Specifically, we …

Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?

W Shen, L Cheng, Y Yang, M Li, Q Zhang - arxiv preprint arxiv …, 2023 - arxiv.org
In this paper, we explain the inference logic of large language models (LLMs) as a set of
symbolic concepts. Many recent studies have discovered that traditional DNNs usually …

Why pre-training is beneficial for downstream classification tasks?

X Jiang, X Cheng, Z Li - arxiv preprint arxiv:2410.08455, 2024 - arxiv.org
Pre-training has exhibited notable benefits to downstream tasks by boosting accuracy and
speeding up convergence, but the exact reasons for these benefits still remain unclear. To …