Tensor networks meet neural networks: A survey and future perspectives
Tensor networks (TNs) and neural networks (NNs) are two fundamental data modeling
approaches. TNs were introduced to solve the curse of dimensionality in large-scale tensors …
approaches. TNs were introduced to solve the curse of dimensionality in large-scale tensors …
Heuristic rank selection with progressively searching tensor ring network
Recently, tensor ring networks (TRNs) have been applied in deep networks, achieving
remarkable successes in compression ratio and accuracy. Although highly related to the …
remarkable successes in compression ratio and accuracy. Although highly related to the …
A unified weight initialization paradigm for tensorial convolutional neural networks
Abstract Tensorial Convolutional Neural Networks (TCNNs) have attracted much research
attention for their power in reducing model parameters or enhancing the generalization …
attention for their power in reducing model parameters or enhancing the generalization …
Preparing Lessons for Progressive Training on Language Models
The rapid progress of Transformers in artificial intelligence has come at the cost of increased
resource consumption and greenhouse gas emissions due to growing model sizes. Prior …
resource consumption and greenhouse gas emissions due to growing model sizes. Prior …
Tednet: A pytorch toolkit for tensor decomposition networks
Abstract Tensor Decomposition Networks (TDNs) prevail for their inherent compact
architectures. To give more researchers a flexible way to exploit TDNs, we present a Pytorch …
architectures. To give more researchers a flexible way to exploit TDNs, we present a Pytorch …