Are large kernels better teachers than transformers for convnets?

T Huang, L Yin, Z Zhang, L Shen… - International …, 2023 - proceedings.mlr.press
This paper reveals a new appeal of the recently emerged large-kernel Convolutional Neural
Networks (ConvNets): as the teacher in Knowledge Distillation (KD) for small-kernel …

Continual learning with deep neural networks in physiological signal data: A survey

A Li, H Li, G Yuan - Healthcare, 2024 - mdpi.com
Deep-learning algorithms hold promise in processing physiological signal data, including
electrocardiograms (ECGs) and electroencephalograms (EEGs). However, healthcare often …

UniTS: A unified multi-task time series model

S Gao, T Koker, O Queen, T Hartvigsen… - The Thirty-eighth …, 2024 - openreview.net
Although pre-trained transformers and reprogrammed text-based LLMs have shown strong
performance on time series tasks, the best-performing architectures vary widely across …

Units: Building a unified time series model

S Gao, T Koker, O Queen, T Hartvigsen… - arxiv preprint arxiv …, 2024 - arxiv.org
Foundation models, especially LLMs, are profoundly transforming deep learning. Instead of
training many task-specific models, we can adapt a single pretrained model to many tasks …

Convtimenet: A deep hierarchical fully convolutional model for multivariate time series analysis

M Cheng, J Yang, T Pan, Q Liu, Z Li - arxiv preprint arxiv:2403.01493, 2024 - arxiv.org
This paper introduces ConvTimeNet, a novel deep hierarchical fully convolutional network
designed to serve as a general-purpose model for time series analysis. The key design of …

Review of Time Series Classification Techniques and Methods

W Mahmud, AZ Fanani, HA Santoso… - … on Application for …, 2023 - ieeexplore.ieee.org
In order to spot trends in the methodologies and procedures employed, this systematic
literature review will look at works on time series categorization. Six research questions are …

Diversify: A General Framework for Time Series Out-of-distribution Detection and Generalization

W Lu, J Wang, X Sun, Y Chen, X Ji… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Time series remains one of the most challenging modalities in machine learning research.
Out-of-distribution (OOD) detection and generalization on time series often face difficulties …

CNN kernels can be the best shapelets

E Qu, Y Wang, X Luo, W He, K Ren… - The Twelfth International …, 2024 - openreview.net
Shapelets and CNN are two typical approaches to model time series. Shapelets aim at
finding a set of sub-sequences that extract feature-based interpretable shapes, but may …

Are Sparse Neural Networks Better Hard Sample Learners?

Q **ao, B Wu, L Yin, CN Gadzinski, T Huang… - arxiv preprint arxiv …, 2024 - arxiv.org
While deep learning has demonstrated impressive progress, it remains a daunting
challenge to learn from hard samples as these samples are usually noisy and intricate …

TSec: An Efficient and Effective Framework for Time Series Classification

Y Yao, H Jie, L Chen, T Li, Y Gao… - 2024 IEEE 40th …, 2024 - ieeexplore.ieee.org
Time series classification assigns predefined labels or classes to sequences of data points
ordered chronologically, which is a fundamental task for time series analysis. Existing time …