Activation functions in deep learning: A comprehensive survey and benchmark

SR Dubey, SK Singh, BB Chaudhuri - Neurocomputing, 2022 - Elsevier
Neural networks have shown tremendous growth in recent years to solve numerous
problems. Various types of neural networks have been introduced to deal with different types …

A survey on modern trainable activation functions

A Apicella, F Donnarumma, F Isgrò, R Prevete - Neural Networks, 2021 - Elsevier
In neural networks literature, there is a strong interest in identifying and defining activation
functions which can improve neural network performance. In recent years there has been a …

SSDMNV2: A real time DNN-based face mask detection system using single shot multibox detector and MobileNetV2

P Nagrath, R Jain, A Madan, R Arora, P Kataria… - Sustainable cities and …, 2021 - Elsevier
Face mask detection had seen significant progress in the domains of Image processing and
Computer vision, since the rise of the Covid-19 pandemic. Many face detection models have …

Hippo: Recurrent memory with optimal polynomial projections

A Gu, T Dao, S Ermon, A Rudra… - Advances in neural …, 2020 - proceedings.neurips.cc
A central problem in learning from sequential data is representing cumulative history in an
incremental fashion as more data is processed. We introduce a general framework (HiPPO) …

How important are activation functions in regression and classification? A survey, performance comparison, and future directions

AD Jagtap, GE Karniadakis - Journal of Machine Learning for …, 2023 - dl.begellhouse.com
Inspired by biological neurons, the activation functions play an essential part in the learning
process of any artificial neural network (ANN) commonly used in many real-world problems …

Accurate deep neural network inference using computational phase-change memory

V Joshi, M Le Gallo, S Haefeli, I Boybat… - Nature …, 2020 - nature.com
In-memory computing using resistive memory devices is a promising non-von Neumann
approach for making energy-efficient deep learning inference hardware. However, due to …

Model compression via distillation and quantization

A Polino, R Pascanu, D Alistarh - arxiv preprint arxiv:1802.05668, 2018 - arxiv.org
Deep neural networks (DNNs) continue to make significant advances, solving tasks from
image classification to translation or reinforcement learning. One aspect of the field receiving …

An integrated mediapipe-optimized GRU model for Indian sign language recognition

B Subramanian, B Olimov, SM Naik, S Kim, KH Park… - Scientific Reports, 2022 - nature.com
Sign language recognition is challenged by problems, such as accurate tracking of hand
gestures, occlusion of hands, and high computational cost. Recently, it has benefited from …

Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions

AD Jagtap, Y Shin, K Kawaguchi, GE Karniadakis - Neurocomputing, 2022 - Elsevier
We propose a new type of neural networks, Kronecker neural networks (KNNs), that form a
general framework for neural networks with adaptive activation functions. KNNs employ the …

Activation functions and their characteristics in deep neural networks

B Ding, H Qian, J Zhou - 2018 Chinese control and decision …, 2018 - ieeexplore.ieee.org
Deep neural networks have gained remarkable achievements in many research areas,
especially in computer vision, and natural language processing. The great successes of …