This paper explores the environmental impact of the super-linear growth trends for AI from a
holistic perspective, spanning Data, Algorithms, and System Hardware. We characterize the …

The advancements in machine learning (ML) opened a new opportunity to bring intelligence
to the low-end Internet-of-Things (IoT) nodes, such as microcontrollers. Conventional ML …

In this paper, we explore a novel knowledge-transfer task, termed as Deep Model
Reassembly (DeRy), for general-purpose model reuse. Given a collection of heterogeneous …

The effectiveness of machine learning algorithms arises from being able to extract useful
features from large amounts of data. As model and dataset sizes increase, dataset …

The time and effort involved in hand-designing deep neural networks is immense. This has
prompted the development of Neural Architecture Search (NAS) techniques to automate this …

In the past decade, advances in deep learning have resulted in breakthroughs in a variety of
areas, including computer vision, natural language understanding, speech recognition, and …

Abstract Knowledge distillation (KD) is an effective training strategy to improve the
lightweight student models under the guidance of cumbersome teachers. However, the large …

Abstract Spiking Neural Networks (SNNs) have gained huge attention as a potential energy-
efficient alternative to conventional Artificial Neural Networks (ANNs) due to their inherent …

In this paper, we present Auto-KD, the first automated search framework for optimal
knowledge distillation design. Traditional distillation techniques typically require handcrafted …

Accuracy predictor is a key component in Neural Architecture Search (NAS) for ranking
architectures. Building a high-quality accuracy predictor usually costs enormous …