NLP-progress README

The project's README from github.com/sebastianruder/NLP-progress. As one markdown file: README.md.

Tracking Progress in Natural Language Processing

Table of contents

English

Vietnamese

Hindi

Chinese

For more tasks, datasets and results in Chinese, check out the Chinese NLP website.

French

Russian

Spanish

Portuguese

Korean

Nepali

Bengali

Persian

Turkish

German

Arabic

This document aims to track the progress in Natural Language Processing (NLP) and give an overview of the state-of-the-art (SOTA) across the most common NLP tasks and their corresponding datasets.

It aims to cover both traditional and core NLP tasks such as dependency parsing and part-of-speech tagging as well as more recent ones such as reading comprehension and natural language inference. The main objective is to provide the reader with a quick overview of benchmark datasets and the state-of-the-art for their task of interest, which serves as a stepping stone for further research. To this end, if there is a place where results for a task are already published and regularly maintained, such as a public leaderboard, the reader will be pointed there.

If you want to find this document again in the future, just go to nlpprogress.com or nlpsota.com in your browser.

Contributing

Guidelines

Results   Results reported in published papers are preferred; an exception may be made for influential preprints.

Datasets   Datasets should have been used for evaluation in at least one published paper besides the one that introduced the dataset.

Code   We recommend to add a link to an implementation if available. You can add a Code column (see below) to the table if it does not exist. In the Code column, indicate an official implementation with Official. If an unofficial implementation is available, use Link (see below). If no implementation is available, you can leave the cell empty.

Adding a new result

If you would like to add a new result, you can just click on the small edit button in the top-right corner of the file for the respective task (see below).

Click on the edit button to add a file

This allows you to edit the file in Markdown. Simply add a row to the corresponding table in the same format. Make sure that the table stays sorted (with the best result on top). After you've made your change, make sure that the table still looks ok by clicking on the "Preview changes" tab at the top of the page. If everything looks good, go to the bottom of the page, where you see the below form.

Fill out the file change information

Add a name for your proposed change, an optional description, indicate that you would like to "Create a new branch for this commit and start a pull request", and click on "Propose file change".

Adding a new dataset or task

For adding a new dataset or task, you can also follow the steps above. Alternatively, you can fork the repository. In both cases, follow the steps below:

  1. If your task is completely new, create a new file and link to it in the table of contents above.
  2. If not, add your task or dataset to the respective section of the corresponding file (in alphabetical order).
  3. Briefly describe the dataset/task and include relevant references.
  4. Describe the evaluation setting and evaluation metric.
  5. Show how an annotated example of the dataset/task looks like.
  6. Add a download link if available.
  7. Copy the below table and fill in at least two results (including the state-of-the-art) for your dataset/task (change Score to the metric of your dataset). If your dataset/task has multiple metrics, add them to the right of Score.
  8. Submit your change as a pull request.
Model Score Paper / Source Code

Wish list

These are tasks and datasets that are still missing:

Exporting into a structured format

You can extract all the data into a structured, machine-readable JSON format with parsed tasks, descriptions and SOTA tables.

The instructions are in structured/README.md.

Instructions for building the site locally

Instructions for building the website locally using Jekyll can be found here.

NLP-progress (sebastianruder/NLP-progress)

Searching Database…

23K ·GitHub
NLP-progress
这是什么？这是一份由社区维护、经过整理的文档，用来跟踪自然语言处理（NLP）的进展。它列出了大多数常见 NLP 任务在多种语言上的标准数据集，以及当前最先进（SOTA）的结果。
这是代码库，还是只是一份参考文档？这是参考文档，不是代码。它是一组 Markdown 文件，每个 NLP 任务一个文件，列出数据集、SOTA 表格（模型、分数、论文、代码链接）以及示例标注。
内容是怎么组织的？我应该从哪里开始？内容先按语言划分（英语、中文、法语、西班牙语等），每种语言下再按任务划分（例如机器翻译、命名实体识别、问答），这些都列在 README 顶部的目录中。选择你感兴趣的任务，然后打开对应的 Markdown 文件即可。
阅读或使用这份列表需要安装什么吗？不需要。它只是可以在 GitHub 或 nlpprogress.com 上阅读的 Markdown 文件；如果要在本地构建网站，需要 Jekyll；将数据导出为 JSON 则要用 structured/README.md 中介绍的另一个脚本。
所有条目都经过审核吗？它还在更新吗？条目依据贡献指南进行审核（结果必须来自已发表的论文，数据集必须至少在另一篇已发表的论文中使用过）。但作为一份社区维护的列表，项目自 2018 年启动，部分任务可能落后于最新研究。
这份资料对谁有用？对于研究人员和从业者来说，在开展自己的工作之前，它能让人快速了解某个特定 NLP 任务的基准数据集和当前的 SOTA 水平。
我能否获得机器可读格式的数据，而不是阅读 Markdown？可以。该仓库支持将所有任务、描述和 SOTA 表格导出为结构化的 JSON 格式，具体步骤见 structured/README.md。
如何向列表中添加新的结果或数据集？直接在 GitHub 上编辑相应任务的 Markdown 文件（或先 fork 仓库），按照贡献指南在 SOTA 表格中添加一行，或描述新的数据集或任务，然后提交 pull request。
除英语外，还涵盖哪些语言？越南语、印地语、中文、法语、俄语、西班牙语、葡萄牙语、韩语、尼泊尔语、孟加拉语、波斯语、土耳其语、德语和阿拉伯语都有各自独立的任务章节。
这份列表是否明确承认有哪些内容缺失？是的。“Wish list”（愿望清单）一节列出了一些空白，例如双语词典归纳、篇章分析、关键短语抽取、知识库填充和半监督学习任务。
What are similar projects?funNLP, awesome-nlp, transferlearning, LLMSearchRecommender, LLMsPracticalGuide, Awesome-LLM, Deep-Learning-Papers-Reading-Roadmap, ParlAI
Which topics is it in?named-entity-recognition, machine-translation, question-answering, sentiment-analysis, papers, natural-language-processing, dataset, academic-research, nlp, benchmark, awesome-list, list
Creator
sebastianruder
Size
1.4 MB
Stars
23K
Updated
CommitJun 23Releasev0.3Mar 26
Age
8y
Commits · 90 Days