Ordered neurons: Integrating tree structures into recurrent neural networks

Y Shen, S Tan, A Sordoni, A Courville - arxiv preprint arxiv:1810.09536, 2018 - arxiv.org
Natural language is hierarchically structured: smaller units (eg, phrases) are nested within
larger units (eg, clauses). When a larger constituent ends, all of the smaller constituents that …

Ordered memory

Y Shen, S Tan, A Hosseini, Z Lin… - Advances in …, 2019 - proceedings.neurips.cc
Stack-augmented recurrent neural networks (RNNs) have been of interest to the deep
learning community for some time. However, the difficulty of training memory models …

Self-instantiated recurrent units with dynamic soft recursion

A Zhang, Y Tay, Y Shen, A Chan… - Advances in Neural …, 2021 - proceedings.neurips.cc
While standard recurrent neural networks explicitly impose a chain structure on different
forms of data, they do not have an explicit bias towards recursive self-instantiation where the …

Dependency-based mixture language models

Z Yang, X Wan - arxiv preprint arxiv:2203.10256, 2022 - arxiv.org
Various models have been proposed to incorporate knowledge of syntactic structures into
neural language models. However, previous works have relied heavily on elaborate …

Neural unsupervised parsing beyond english

K Kann, A Mohananey, S Bowman… - Proceedings of the 2nd …, 2019 - aclanthology.org
Recently, neural network models which automatically infer syntactic structure from raw text
have started to achieve promising results. However, earlier work on unsupervised parsing …

Assessing incrementality in sequence-to-sequence models

D Ulmer, D Hupkes, E Bruni - arxiv preprint arxiv:1906.03293, 2019 - arxiv.org
Since their inception, encoder-decoder models have successfully been applied to a wide
array of problems in computational linguistics. The most recent successes are predominantly …

Recursive top-down production for sentence generation with latent trees

S Tan, Y Shen, TJ O'Donnell, A Sordoni… - arxiv preprint arxiv …, 2020 - arxiv.org
We model the recursive production property of context-free grammars for natural and
synthetic languages. To this end, we present a dynamic programming algorithm that …

FastTrees: Parallel Latent Tree-Induction for Faster Sequence Encoding

BTW Pung, A Chan - arxiv preprint arxiv:2111.14031, 2021 - arxiv.org
Inducing latent tree structures from sequential data is an emerging trend in the NLP research
landscape today, largely popularized by recent methods such as Gumbel LSTM and …

Syntactic Inductive Biases for Deep Learning Methods

Y Shen - arxiv preprint arxiv:2206.04806, 2022 - arxiv.org
In this thesis, we try to build a connection between the two schools by introducing syntactic
inductive biases for deep learning models. We propose two families of inductive biases, one …

Length Generalization with Recursive Neural Networks and Beyond

JR Chowdhury - 2024 - search.proquest.com
Abstract We investigate Recursive Neural Networks (RvNNs) for language processing tasks.
Roughly, from a generalized perspective, RvNNs repeatedly apply some neural function on …