A Practitioner's Guide to Continual Multimodal Pretraining

K Roth, V Udandarao, S Dziadzio, A Prabhu… - arxiv preprint arxiv …, 2024 - arxiv.org
Multimodal foundation models serve numerous applications at the intersection of vision and
language. Still, despite being pretrained on extensive data, they become outdated over time …

Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation

H Yan, L Wang, K Ma, Y Zhong - Proceedings of the IEEE …, 2024 - openaccess.thecvf.com
To accommodate real-world dynamics artificial intelligence systems need to cope with
sequentially arriving content in an online manner. Beyond regular Continual Learning (CL) …

[PDF][PDF] Approximate Bayesian Class-Conditional Models under Continuous Representation Shift

TL Lee, A Storkey - International Conference on Artificial …, 2024 - proceedings.mlr.press
For models consisting of a classifier in some representation space, learning online from a
non-stationary data stream often necessitates changes in the representation. So, the …

Similarity-Based Adaptation for Task-Aware and Task-Free Continual Learning

T Adel - Journal of Artificial Intelligence Research, 2024 - jair.org
Continual learning (CL) is a paradigm which addresses the issue of how to learn from
sequentially arriving tasks. The goal of this paper is to introduce a CL framework which can …

CMCN: Chinese medical concept normalization using continual learning and knowledge-enhanced

P Han, X Li, Z Zhang, Y Zhong, L Gu, Y Hua… - Artificial Intelligence in …, 2024 - Elsevier
Abstract Medical Concept Normalization (MCN) is a crucial process for deep information
extraction and natural language processing tasks, which plays a vital role in biomedical …

Dual-CBA: Improving Online Continual Learning via Dual Continual Bias Adaptors from a Bi-level Optimization Perspective

Q Wang, R Wang, Y Wu, X Jia, M Zhou… - arxiv preprint arxiv …, 2024 - arxiv.org
In online continual learning (CL), models trained on changing distributions easily forget
previously learned knowledge and bias toward newly received tasks. To address this issue …

Accelerating Heterogeneous Federated Learning with Closed-form Classifiers

E Fanì, R Camoriano, B Caputo, M Ciccone - arxiv preprint arxiv …, 2024 - arxiv.org
Federated Learning (FL) methods often struggle in highly statistically heterogeneous
settings. Indeed, non-IID data distributions cause client drift and biased local solutions …

[HTML][HTML] A Survey of Continual Learning with Deep Networks: Theory, Method and Application

Z Dongyang, LU Zixuan, LIU Junmin, LI Lanyu - 电子与信息学报, 2024 - jeit.ac.cn
Biological organisms in nature are required to continuously learn from and adapt to the
environment throughout their lifetime. This ongoing learning capacity serves as the …

What Matters in Graph Class Incremental Learning? An Information Preservation Perspective

J Li, Y Wang, P Zhu, W Lin, Q Hu - The Thirty-eighth Annual Conference on … - openreview.net
Graph class incremental learning (GCIL) requires the model to classify emerging nodes of
new classes while remembering old classes. Existing methods are designed to preserve …