Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 77편

CMU CoNaLa, the Code/Natural Language Challenge

CoNaLa
CoNaLa 설명 보기

The CMU CoNaLa, the Code/Natural Language Challenge dataset is a joint project from the Carnegie Mellon University NeuLab and Strudel labs. Its purpose is for testing the generation of code snippets from natural language. The data comes from StackOverflow questions. There are 2379 training and 500 test examples that were manually annotated. Every example has a natural language intent and its corresponding python snippet. In addition to the manually annotated dataset, there are also 598,237 mined intent-snippet pairs. These examples are similar to the hand-annotated ones except that they contain a probability if the pair is valid. Source: CoNaLa dataset Homepage

관련 논문 10편

Multilingual CoNaLa

MCoNaLa
MCoNaLa 설명 보기

MCoNaLa is a multilingual dataset to benchmark code generation from natural language commands extending beyond English. Modeled off of the methodology from the English Code/Natural Language Challenge (CoNALa) dataset, the authors annotated a total of 896 NL-code pairs in three languages: Spanish, Japanese, and Russian. Due to the limited sample of multiple languages, we use English CoNaLa samples for training, where the intents are originally written in English. Spanish, Japanese, and Russian are of the Target Language (TL), whose samples are always (only) used for testing purpose due to the limited amount. English is the High-Resource Language (HRL) for which the samples can be leveraged for model training.

관련 논문 4편

CoNaLa Extended With Question Text

CoNaLa-Ext
CoNaLa-Ext 설명 보기

The CoNaLa Extended With Question Text is an extension to the original CoNaLa Dataset (Papers With Code Link) proposed in the NLP4Prog workshop paper "Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation". The key additions are that every example now has the full question body from its respective StackOverflow Question. IMPORTANT If you use this dataset, you MUST cite the original CoNaLa dataset paper. Source: CoNaLa-Ext Homepage