GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow

Read Full Paperexternally

Key Points

Key points are not available for this paper at this time.

Abstract

GPT-Neo is an implementation of model & data-parallel GPT-2 and GPT-3-like models, utilizing Mesh Tensorflow for distributed support. This codebase is designed for TPUs. It should also work on GPUs, though we do not recommend this hardware configuration.

Connected Papers

Building similarity graph...

Analyzing shared references across papers

Discussion

Authors

Sid Black

Leo Gao

Booz Allen Hamilton (United States)

Phil Wang

Actions

References and Citations

Connected Papers

Building similarity graph...

Analyzing shared references across papers

GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow

Key Points

Abstract

Citation Network

Connected Papers

Discussion

Authors

Actions

References and Citations

Citation Network

Connected Papers

Discussion

Cite this study