Normalization techniques are essential for accelerating the training and improving the
generalization of deep neural networks (DNNs), and have successfully been used in various …

Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional
data, due to their simple geometric interpretations and typically attractive computational …

Orthogonal matrix has shown advantages in training Recurrent Neural Networks (RNNs),
but such matrix is limited to be square for the hidden-to-hidden transformation in RNNs. In …

In this paper, we first propose a novel generalized power iteration (GPI) method to solve the
quadratic problem on the Stiefel manifold (QPSM) as min_ W^ TW= I min WTW= I Tr (WT …

The problem of optimizing a quadratic form over an orthogonality constraint (QP-OC for
short) is one of the most fundamental matrix optimization problems and arises in many …

A Stiefel manifold of the compact type is often encountered in many fields of engineering
including, signal and image processing, machine learning, numerical optimization and …

The Kuramoto model of coupled phase oscillators is often used to describe synchronization
phenomena in nature. Some applications, eg, quantum synchronization and rigid-body …

The aim of the present tutorial paper is to recall notions from manifold calculus and to
illustrate how these tools prove useful in describing system-theoretic properties. Special …

We propose a new class of Riemannian conjugate gradient (CG) methods, in which inverse
retraction is used instead of vector transport for search direction construction. In existing …

The symmetric eigenvalue decomposition and the singular value decomposition (SVD) are
fundamental matrix decompositions with many applications. Conventional algorithms for …