A comprehensive survey of dataset distillation

S Lei, D Tao - IEEE Transactions on Pattern Analysis and …, 2023 - ieeexplore.ieee.org
Deep learning technology has developed unprecedentedly in the last decade and has
become the primary choice in many application domains. This progress is mainly attributed …

Dataset quantization

D Zhou, K Wang, J Gu, X Peng, D Lian… - Proceedings of the …, 2023 - openaccess.thecvf.com
State-of-the-art deep neural networks are trained with large amounts (millions or even
billions) of data. The expensive computation and memory costs make it difficult to train them …

Preventing zero-shot transfer degradation in continual learning of vision-language models

Z Zheng, M Ma, K Wang, Z Qin… - Proceedings of the …, 2023 - openaccess.thecvf.com
Continual learning (CL) can help pre-trained vision-language models efficiently adapt to
new or under-trained data distributions without re-training. Nevertheless, during the …

Towards lossless dataset distillation via difficulty-aligned trajectory matching

Z Guo, K Wang, G Cazenavette, H Li, K Zhang… - arxiv preprint arxiv …, 2023 - arxiv.org
The ultimate goal of Dataset Distillation is to synthesize a small synthetic dataset such that a
model trained on this synthetic set will perform equally well as a model trained on the full …

Does graph distillation see like vision dataset counterpart?

B Yang, K Wang, Q Sun, C Ji, X Fu… - Advances in …, 2023 - proceedings.neurips.cc
Training on large-scale graphs has achieved remarkable results in graph representation
learning, but its cost and storage have attracted increasing concerns. Existing graph …

Efficient dataset distillation via minimax diffusion

J Gu, S Vahidian, V Kungurtsev… - Proceedings of the …, 2024 - openaccess.thecvf.com
Dataset distillation reduces the storage and computational consumption of training a
network by generating a small surrogate dataset that encapsulates rich information of the …

Generalized large-scale data condensation via various backbone and statistical matching

S Shao, Z Yin, M Zhou, X Zhang… - Proceedings of the …, 2024 - openaccess.thecvf.com
The lightweight" local-match-global" matching introduced by SRe2L successfully creates a
distilled dataset with comprehensive information on the full 224x224 ImageNet-1k. However …

You only condense once: Two rules for pruning condensed datasets

Y He, L **ao, JT Zhou - Advances in Neural Information …, 2023 - proceedings.neurips.cc
Dataset condensation is a crucial tool for enhancing training efficiency by reducing the size
of the training dataset, particularly in on-device scenarios. However, these scenarios have …

M3d: Dataset condensation by minimizing maximum mean discrepancy

H Zhang, S Li, P Wang, D Zeng, S Ge - Proceedings of the AAAI …, 2024 - ojs.aaai.org
Training state-of-the-art (SOTA) deep models often requires extensive data, resulting in
substantial training and storage costs. To address these challenges, dataset condensation …

Navigating complexity: Toward lossless graph condensation via expanding window matching

Y Zhang, T Zhang, K Wang, Z Guo, Y Liang… - arxiv preprint arxiv …, 2024 - arxiv.org
Graph condensation aims to reduce the size of a large-scale graph dataset by synthesizing
a compact counterpart without sacrificing the performance of Graph Neural Networks …