gpt-2 README

The project's README from github.com/openai/gpt-2. As one markdown file: README.md.

Status: Archive (code is provided as-is, no updates expected)

gpt-2

Code and models from the paper "Language Models are Unsupervised Multitask Learners".

You can read about GPT-2 and its staged release in our original blog post, 6 month follow-up post, and final post.

We have also released a dataset for researchers to study their behaviors.

* Note that our original parameter counts were wrong due to an error (in our previous blog posts and paper). Thus you may have seen small referred to as 117M and medium referred to as 345M.

Usage

This repository is meant to be a starting point for researchers and engineers to experiment with GPT-2.

For basic information, see our model card.

Some caveats

Work with us

Please let us know if you’re doing interesting research with or working on applications of GPT-2! We’re especially interested in hearing from and potentially working with those who are studying

Development

See DEVELOPERS.md

Contributors

See CONTRIBUTORS.md

Citation

Please use the following bibtex entry:

@article{radford2019language,
  title={Language Models are Unsupervised Multitask Learners},
  author={Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya},
  year={2019}
}

Future work

We may release code for evaluating the models on various benchmarks.

We are still considering release of the larger models.

License

Modified MIT

gpt-2 (openai/gpt-2)

Searching Database…

25K ·GitHub
archived
这是什么?这是 OpenAI 官方的 GPT-2 代码与发布版本。GPT-2 是论文《Language Models are Unsupervised Multitask Learners》中所描述的语言模型。它包含模型代码,以及用于从已发布的 GPT-2 检查点生成文本样本的脚本。
这是现成的应用,还是只是研究代码?这是一个研究代码库,并不是成熟的产品。README 将其定位为用于试验模型的起点,而不是带用户界面的软件。
我该怎么把它跑起来?主 README 将环境和安装细节交给单独的 DEVELOPERS.md 文件。完成设置后,针对已发布的 GPT-2 检查点运行所附的采样脚本,即可生成文本。
我需要单独下载模型权重吗?需要。你需要已发布的 GPT-2 检查点,以及一个 Python 环境。README 还指出,最初的参数量标注有误,因此小模型应为 117M,中等模型应为 345M。
这个项目还在维护吗?不再维护。项目已归档,以现状冻结,也不计划再更新。README 还提醒,GPT-2 的稳健性尚未被充分理解,其输出可能存在细微的不连贯、偏见或不准确之处,因此样本应明确标注为合成内容。
为什么 OpenAI 建议将输出标注为合成内容?因为 GPT-2 生成的样本可能存在“细微之处不连贯或不准确”的问题,需要仔细阅读才能察觉。因此 OpenAI 要求,在广泛分享前应明确将生成文本标记为合成内容,以免被误认为是人类写作。
OpenAI 对哪些后续研究感兴趣?OpenAI 表示,希望听取研究恶意使用场景的人员的意见,例如检测合成文本的研究者;也希望听取研究模型中固有偏见和问题内容的人员的意见。
较大的 GPT-2 模型后来有被加入这个仓库吗?README 写道,OpenAI“仍在考虑发布更大的模型”,并可能之后再加入基准评测代码。因此,最初分阶段发布的模型并不保证都会出现在这里。
What are similar projects?gpt-3, minGPT, fairseq, nanoGPT, bert, math, bdh, train-llm-from-scratch
Creator
openai
Size
4.4 MB
Stars
25K
Updated
CommitJan 26
Age
7y
Commits · 90 Days