What does this research mean for the field?

Properly tuned standard LSTM and QRNN architectures can achieve state-of-the-art results on both character-level and word-level language modeling tasks, demonstrating that highly complex or specialized architectures are not strictly necessary for top performance. Novelty: ClaimNovelty.INCREMENTAL. Consensus alignment: ConsensusAlignment.NEUTRAL.

March 22, 2018Open Access

An Analysis of Neural Language Modeling at Multiple Scales

Puntos clave

Los puntos clave no están disponibles para este artículo en este momento.

Resumen

Many of the leading approaches in language modeling introduce novel, complex and specialized architectures. We take existing state-of-the-art word level language models based on LSTMs and QRNNs and extend them to both larger vocabularies as well as character-level granularity. When properly tuned, LSTMs and QRNNs achieve state-of-the-art results on character-level (Penn Treebank, enwik8) and word-level (WikiText-103) datasets, respectively. Results are obtained in only 12 hours (WikiText-103) to 2 days (enwik8) using a single modern GPU.

Me gusta

Guardar

Ver artículo completo