CorrSc: 0.848
Papers with Code · CC BY-SA 4.0
Code Generation
Code Generation is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.
<span class="description-source">Source: Deep Learning for Source Code Modeling and Generation </span>
Image source: Measuring Coding Challenge Competence With APPS
논문
BLEU: 20.54
BLEU: 35.32
Execution Accuracy: 74.6Exact Match Accuracy: 61.0
Execution Accuracy: 59.4Exact Match Accuracy: 48.3
easy pass@1: 31.50%
BLEU: 18.85
Execution Accuracy: 82.6
Execution Accuracy: 62.5Exact Match Accuracy: 69
Execution Accuracy: 89.2Exact Match Accuracy: 83.7
Pass@1: 87.19
Pass Rate: 42.5w/o Intact: 42.5w/o PE: 62.5
Execution Accuracy: 68.0Exact Match Accuracy: 62.8
Pass@1: 65.8
BLEU: 19.85