Attention networks in multimodal learning provide an efficient way to utilize given visual
information selectively. However, the computational cost to learn attention distributions for …

In this paper we describe a new mobile architecture MobileNetV2 that improves the state of
the art performance of mobile models on multiple benchmarks across a spectrum of different …

We introduce a novel technique for knowledge transfer, where knowledge from a pretrained
deep neural network (DNN) is distilled and transferred to another DNN. As the DNN …

State-of-the-art approaches for semantic image segmentation are built on Convolutional
Neural Networks (CNNs). The typical segmentation architecture is composed of (a) a …

Convolutional neural networks (CNNs) exhibit good performance in image processing tasks,
pointing themselves as the current state-of-the-art of deep learning methods. However, the …

We propose the Gaussian Error Linear Unit (GELU), a high-performing neural network
activation function. The GELU activation function is $ x\Phi (x) $, where $\Phi (x) $ the …

Deep neural networks have become the state-of-the-art models in numerous machine
learning tasks. However, general guidance to network architecture design is still missing. In …

Deep convolutional neural networks (DCNNs) have shown remarkable performance in
image classification tasks in recent years. Generally, deep neural network architectures are …

Neuroscience has focused on the detailed implementation of computation, studying neural
codes, dynamics and circuits. In machine learning, however, artificial neural networks tend …

Deep learning-based algorithms have been widely employed to build reliable steel surface
defect detection systems, which are important for manufacturing. The performance of deep …