Open Pre-trained Transformer Language Models
lastmod 2022-05-06created 2022-05-061 edits
A GPT-3-scale (175B) language model released by Meta
Features
- Large language models like this take considerable capital to build, and releasing only API access makes them hard to research
- For interested researchers, the full model, experiment code, and logbook are all released for download
- Needs only 1/7 the Carbon Footprint of GPT-3
Documentations
- OPT - Open Pre-trained Transformer Language Models
- https://arxiv.org/abs/2205.01068
See also
Favorite site
- Github - metaseq - A codebase for working with Open Pre-trained Transformers.