A unified approach to domain incremental learning with memory: Theory and algorithm

H Shi, H Wang - Advances in Neural Information Processing …, 2023 - proceedings.neurips.cc
Abstract Domain incremental learning aims to adapt to a sequence of domains with access
to only a small subset of data (ie, memory) from previous domains. Various methods have …

Deep neural collapse is provably optimal for the deep unconstrained features model

P Súkeník, M Mondelli… - Advances in Neural …, 2023 - proceedings.neurips.cc
Neural collapse (NC) refers to the surprising structure of the last layer of deep neural
networks in the terminal phase of gradient descent training. Recently, an increasing amount …

Compressible dynamics in deep overparameterized low-rank learning & adaptation

C Yaras, P Wang, L Balzano, Q Qu - arxiv preprint arxiv:2406.04112, 2024 - arxiv.org
While overparameterization in machine learning models offers great benefits in terms of
optimization and generalization, it also leads to increased computational requirements as …

Generalized neural collapse for a large number of classes

J Jiang, J Zhou, P Wang, Q Qu, D Mixon, C You… - arxiv preprint arxiv …, 2023 - arxiv.org
Neural collapse provides an elegant mathematical characterization of learned last layer
representations (aka features) and classifier weights in deep classification models. Such …

Neural collapse in deep linear networks: from balanced to imbalanced data

H Dang, TT Huu, S Osher, N Ho, TM Nguyen - 2023 - openreview.net
Modern deep neural networks have achieved impressive performance on tasks from image
classification to natural language processing. Surprisingly, these complex systems with …

Principled and efficient transfer learning of deep models via neural collapse

X Li, S Liu, J Zhou, X Lu, C Fernandez-Granda… - arxiv e …, 2022 - ui.adsabs.harvard.edu
As model size continues to grow and access to labeled training data remains limited,
transfer learning has become a popular approach in many scientific and engineering fields …

YOLO-adaptor: a fast adaptive one-stage detector for non-aligned visible-infrared object detection

H Fu, H Liu, J Yuan, X He, J Lin… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Visible-infrared object detection has attracted increasing attention recently due to its
superior performance and cost-efficiency. Most existing methods focus on the detection of …

Understanding deep representation learning via layerwise feature compression and discrimination

P Wang, X Li, C Yaras, Z Zhu, L Balzano, W Hu… - arxiv preprint arxiv …, 2023 - arxiv.org
Over the past decade, deep learning has proven to be a highly effective tool for learning
meaningful features from raw data. However, it remains an open question how deep …

Navigate beyond shortcuts: Debiased learning through the lens of neural collapse

Y Wang, J Sun, C Wang, M Zhang… - Proceedings of the …, 2024 - openaccess.thecvf.com
Recent studies have noted an intriguing phenomenon termed Neural Collapse that is when
the neural networks establish the right correlation between feature spaces and the training …

The law of parsimony in gradient descent for learning deep linear networks

C Yaras, P Wang, W Hu, Z Zhu, L Balzano… - arxiv preprint arxiv …, 2023 - arxiv.org
Over the past few years, an extensively studied phenomenon in training deep networks is
the implicit bias of gradient descent towards parsimonious solutions. In this work, we …