A brain-inspired algorithm that mitigates catastrophic forgetting of artificial and spiking neural networks with low computational cost

T Zhang, X Cheng, S Jia, CT Li, M Poo, B Xu - Science Advances, 2023 - science.org
Neuromodulators in the brain act globally at many forms of synaptic plasticity, represented
as metaplasticity, which is rarely considered by existing spiking (SNNs) and nonspiking …

Holomorphic equilibrium propagation computes exact gradients through finite size oscillations

A Laborieux, F Zenke - Advances in neural information …, 2022 - proceedings.neurips.cc
Equilibrium propagation (EP) is an alternative to backpropagation (BP) that allows the
training of deep neural networks with local learning rules. It thus provides a compelling …

On training implicit models

Z Geng, XY Zhang, S Bai, Y Wang… - Advances in Neural …, 2021 - proceedings.neurips.cc
This paper focuses on training implicit models of infinite layers. Specifically, previous works
employ implicit differentiation and solve the exact gradient for the backward propagation …

Gradients without backpropagation

AG Baydin, BA Pearlmutter, D Syme, F Wood… - arxiv preprint arxiv …, 2022 - arxiv.org
Using backpropagation to compute gradients of objective functions for optimization has
remained a mainstay of machine learning. Backpropagation, or reverse-mode differentiation …

Self-backpropagation of synaptic modifications elevates the efficiency of spiking and artificial neural networks

T Zhang, X Cheng, S Jia, M Poo, Y Zeng, B Xu - Science advances, 2021 - science.org
Many synaptic plasticity rules found in natural circuits have not been incorporated into
artificial neural networks (ANNs). We showed that incorporating a nonlocal feature of …

Bottom-up and top-down approaches for the design of neuromorphic processing systems: tradeoffs and synergies between natural and artificial intelligence

C Frenkel, D Bol, G Indiveri - Proceedings of the IEEE, 2023 - ieeexplore.ieee.org
While Moore's law has driven exponential computing power expectations, its nearing end
calls for new avenues for improving the overall system performance. One of these avenues …

Information Geometry? Exercises de Styles

IA Mageed, KQ Zhang - electronic Journal of Computer Science …, 2022 - ejcsit.uniten.edu.my
Indeed, information geometry, IG is an ever-growing area with a great scope of applications
ranging from Probability & Statistics, Machine Learning (ML), Artificial Intelligence (AI) …

Inferring neural activity before plasticity as a foundation for learning beyond backpropagation

Y Song, B Millidge, T Salvatori, T Lukasiewicz… - Nature …, 2024 - nature.com
For both humans and machines, the essence of learning is to pinpoint which components in
its information processing pipeline are responsible for an error in its output, a challenge that …

The least-control principle for local learning at equilibrium

A Meulemans, N Zucchet… - Advances in …, 2022 - proceedings.neurips.cc
Equilibrium systems are a powerful way to express neural computations. As special cases,
they include models of great current interest in both neuroscience and machine learning …

Bottom-up and top-down neural processing systems design: Neuromorphic intelligence as the convergence of natural and artificial intelligence

CP Frenkel, D Bol, G Indiveri - Ar**v. org, 2021 - zora.uzh.ch
While Moore's law has driven exponential computing power expectations, its nearing end
calls for new avenues for improving the overall system performance. One of these avenues …