The Importance of Being Recurrent for Modeling Hierarchical Structure

Recent work has shown that recurrent neural networks (RNNs) can implicitly capture and exploit hierarchical information when trained to solve common natural language processing tasks In contrast, the ability to model structured data with non-recurrent neural networks has received little attention despite their success in many NLP tasks In this work, we compare the two architectures-recurrent versus non-recurrent-with respect to their ability to model hierarchical structure and find that recurrency is indeed important for this purpose. The code and data used in our experiments is available at https: //github. com/ ketranm/fanᵥsᵣnn