Normalization techniques are essential for accelerating the training and improving the
generalization of deep neural networks (DNNs), and have successfully been used in various …

Abstract Invariant Risk Minimization (IRM) is an emerging invariant feature extracting
technique to help generalization with distributional shift. However, we find that there exists a …

We propose a simple but effective multi-source domain generalization technique based on
deep neural networks by incorporating optimized normalization layers that are specific to …

Attention mechanisms have become ubiquitous in NLP. Recent architectures, notably the
Transformer, learn powerful context-aware word representations through layered, multi …

Single domain generalization aims to learn a model that performs well on many unseen
domains with only one domain data for training. Existing works focus on studying the …

Due to the nonlinear modeling capabilities, deep learning prediction networks have become
widely used for smart agriculture. Because the sensing data has noise and complex …

Batch Normalization (BN) improves both convergence and generalization in training neural
networks. This work understands these phenomena theoretically. We analyze BN by using a …

We address a learning-to-normalize problem by proposing Switchable Normalization (SN),
which learns to select different normalizers for different normalization layers of a deep neural …

Batch Normalization (BN) has become an out-of-box technique to improve deep network
training. However, its effectiveness is limited for micro-batch training, ie, each GPU typically …

A well-known issue of Batch Normalization is its significantly reduced effectiveness in the
case of small mini-batch sizes. When a mini-batch contains few examples, the statistics upon …