A brief overview of unsupervised neural speech representation learning

L Borgholt, JD Havtorn, J Edin, L Maaløe… - arxiv preprint arxiv …, 2022 - arxiv.org
Unsupervised representation learning for speech processing has matured greatly in the last
few years. Work in computer vision and natural language processing has paved the way, but …

Robust training of vector quantized bottleneck models

A Łańcucki, J Chorowski, G Sanchez… - … Joint Conference on …, 2020 - ieeexplore.ieee.org
In this paper we demonstrate methods for reliable and efficient training of discrete
representation using Vector-Quantized Variational Auto-Encoder models (VQ-VAEs) …

Word segmentation on discovered phone units with dynamic programming and self-supervised scoring

H Kamper - IEEE/ACM Transactions on Audio, Speech, and …, 2022 - ieeexplore.ieee.org
Recent work on unsupervised speech segmentation has used self-supervised models with
phone and word segmentation modules that are trained jointly. This paper instead revisits …

Variable-rate hierarchical CPC leads to acoustic unit discovery in speech

S Cuervo, A Lancucki, R Marxer… - Advances in …, 2022 - proceedings.neurips.cc
The success of deep learning comes from its ability to capture the hierarchical structure of
data by learning high-level representations defined in terms of low-level ones. In this paper …

Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networks

H Kamper, B van Niekerk - arxiv preprint arxiv:2012.07551, 2020 - arxiv.org
We investigate segmenting and clustering speech into low-bitrate phone-like sequences
without supervision. We specifically constrain pretrained self-supervised vector-quantized …

Unsupervised speech segmentation and variable rate representation learning using segmental contrastive predictive coding

S Bhati, J Villalba, P Żelasko… - … on Audio, Speech …, 2022 - ieeexplore.ieee.org
Typically, unsupervised segmentation of speech into the phone-and wordlike units are
treated as separate tasks and are often done via different methods which do not fully …

Aligned contrastive predictive coding

J Chorowski, G Ciesielski, J Dzikowski… - arxiv preprint arxiv …, 2021 - arxiv.org
We investigate the possibility of forcing a self-supervised model trained using a contrastive
predictive loss to extract slowly varying latent representations. Rather than producing …

Hierarchical residual learning based vector quantized variational autoencorder for image reconstruction and generation

M Adiban, M Siniscalchi, K Stefanov… - 33rd British Machine …, 2022 - diva-portal.org
We propose a multi-layer variational autoencoder method, we call HR-VQVAE, that learns
hierarchical discrete representations of the data. By utilizing a novel objective function, each …

Investigation of process history and underlying phenomena associated with the synthesis of plutonium oxides using Vector Quantizing Variational Autoencoder

CM Hainje, CA Nizinski, SW Jackson, RA Clark… - Chemometrics and …, 2023 - Elsevier
Accurate, high-throughput, and unbiased analysis of plutonium oxide particles is necessary
for analysis of the underlying phenomena associated with the process parameters involved …

The “scribblelens” dutch historical handwriting corpus

HJGA Dolfing, J Bellegarda… - … on frontiers in …, 2020 - ieeexplore.ieee.org
Historical handwritten documents guard an important part of human knowledge only at the
reach of a few scholars and experts. Recent developments in machine learning have the …