Ordered neurons: Integrating tree structures into recurrent neural networks
Natural language is hierarchically structured: smaller units (eg, phrases) are nested within
larger units (eg, clauses). When a larger constituent ends, all of the smaller constituents that …
larger units (eg, clauses). When a larger constituent ends, all of the smaller constituents that …
Ordered memory
Stack-augmented recurrent neural networks (RNNs) have been of interest to the deep
learning community for some time. However, the difficulty of training memory models …
learning community for some time. However, the difficulty of training memory models …
Self-instantiated recurrent units with dynamic soft recursion
While standard recurrent neural networks explicitly impose a chain structure on different
forms of data, they do not have an explicit bias towards recursive self-instantiation where the …
forms of data, they do not have an explicit bias towards recursive self-instantiation where the …
Dependency-based mixture language models
Various models have been proposed to incorporate knowledge of syntactic structures into
neural language models. However, previous works have relied heavily on elaborate …
neural language models. However, previous works have relied heavily on elaborate …
Neural unsupervised parsing beyond english
Recently, neural network models which automatically infer syntactic structure from raw text
have started to achieve promising results. However, earlier work on unsupervised parsing …
have started to achieve promising results. However, earlier work on unsupervised parsing …
Assessing incrementality in sequence-to-sequence models
Since their inception, encoder-decoder models have successfully been applied to a wide
array of problems in computational linguistics. The most recent successes are predominantly …
array of problems in computational linguistics. The most recent successes are predominantly …
Recursive top-down production for sentence generation with latent trees
We model the recursive production property of context-free grammars for natural and
synthetic languages. To this end, we present a dynamic programming algorithm that …
synthetic languages. To this end, we present a dynamic programming algorithm that …
FastTrees: Parallel Latent Tree-Induction for Faster Sequence Encoding
Inducing latent tree structures from sequential data is an emerging trend in the NLP research
landscape today, largely popularized by recent methods such as Gumbel LSTM and …
landscape today, largely popularized by recent methods such as Gumbel LSTM and …
Syntactic Inductive Biases for Deep Learning Methods
Y Shen - arxiv preprint arxiv:2206.04806, 2022 - arxiv.org
In this thesis, we try to build a connection between the two schools by introducing syntactic
inductive biases for deep learning models. We propose two families of inductive biases, one …
inductive biases for deep learning models. We propose two families of inductive biases, one …
Length Generalization with Recursive Neural Networks and Beyond
JR Chowdhury - 2024 - search.proquest.com
Abstract We investigate Recursive Neural Networks (RvNNs) for language processing tasks.
Roughly, from a generalized perspective, RvNNs repeatedly apply some neural function on …
Roughly, from a generalized perspective, RvNNs repeatedly apply some neural function on …