Understanding plasticity in neural networks

C Lyle, Z Zheng, E Nikishin, BA Pires… - International …, 2023 - proceedings.mlr.press
Plasticity, the ability of a neural network to quickly change its predictions in response to new
information, is essential for the adaptability and robustness of deep reinforcement learning …

The dormant neuron phenomenon in deep reinforcement learning

G Sokar, R Agarwal, PS Castro… - … Conference on Machine …, 2023 - proceedings.mlr.press
In this work we identify the dormant neuron phenomenon in deep reinforcement learning,
where an agent's network suffers from an increasing number of inactive neurons, thereby …

The primacy bias in deep reinforcement learning

E Nikishin, M Schwarzer, P D'Oro… - International …, 2022 - proceedings.mlr.press
This work identifies a common flaw of deep reinforcement learning (RL) algorithms: a
tendency to rely on early interactions and ignore useful evidence encountered later …

For sale: State-action representation learning for deep reinforcement learning

S Fujimoto, WD Chang, E Smith… - Advances in neural …, 2023 - proceedings.neurips.cc
In reinforcement learning (RL), representation learning is a proven tool for complex image-
based tasks, but is often overlooked for environments with low-level states, such as physical …

Loss of plasticity in deep continual learning

S Dohare, JF Hernandez-Garcia, Q Lan, P Rahman… - Nature, 2024 - nature.com
Artificial neural networks, deep-learning methods and the backpropagation algorithm form
the foundation of modern machine learning and artificial intelligence. These methods are …

Loss of plasticity in continual deep reinforcement learning

Z Abbas, R Zhao, J Modayil, A White… - … on lifelong learning …, 2023 - proceedings.mlr.press
In this paper, we characterize the behavior of canonical value-based deep reinforcement
learning (RL) approaches under varying degrees of non-stationarity. In particular, we …

Deep reinforcement learning with plasticity injection

E Nikishin, J Oh, G Ostrovski, C Lyle… - Advances in …, 2023 - proceedings.neurips.cc
A growing body of evidence suggests that neural networks employed in deep reinforcement
learning (RL) gradually lose their plasticity, the ability to learn from new data; however, the …

Stop regressing: Training value functions via classification for scalable deep rl

J Farebrother, J Orbay, Q Vuong, AA Taïga… - arxiv preprint arxiv …, 2024 - arxiv.org
Value functions are a central component of deep reinforcement learning (RL). These
functions, parameterized by neural networks, are trained using a mean squared error …

Reincarnating reinforcement learning: Reusing prior computation to accelerate progress

R Agarwal, M Schwarzer, PS Castro… - Advances in neural …, 2022 - proceedings.neurips.cc
Learning tabula rasa, that is without any prior knowledge, is the prevalent workflow in
reinforcement learning (RL) research. However, RL systems, when applied to large-scale …

Disentangling the causes of plasticity loss in neural networks

C Lyle, Z Zheng, K Khetarpal, H van Hasselt… - arxiv preprint arxiv …, 2024 - arxiv.org
Underpinning the past decades of work on the design, initialization, and optimization of
neural networks is a seemingly innocuous assumption: that the network is trained on a\textit …