Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 15편

xCodeEval

xCodeEval 설명 보기

xCodeEval is one of the largest executable multilingual multitask benchmarks consisting of 17 programming languages with execution-level parallelism. It features a total of seven tasks involving code understanding, generation, translation, and retrieval, and it employs an execution-based evaluation instead of traditional lexical approaches. It also provides a test-case-based multilingual code execution engine, ExecEval that supports all the programming languages in xCodeEval.

관련 논문 5편

ComplexCodeEval

ComplexCodeEval 설명 보기

ComplexCodeEval ComplexCodeEval is an evaluation benchmark designed to accommodate multiple downstream tasks, accurately reflect different programming environments, and deliberately avoid data leakage issues. This benchmark includes a diverse set of samples from real-world projects, aiming to closely mirror actual development scenarios. ## Overview ComplexCodeEval consists of: - 3,897 Java samples from 1,055 code repositories - 7,184 Python samples from 2,107 code repositories ### Key Features 1. Diverse Downstream Tasks: The benchmark supports multiple downstream tasks to evaluate the performance of different code analysis tools and models. 2. Accurate Reflection of Programming Environments: Samples are selected from projects that use popular third-party frameworks and packages. 3. Avoidance of Data Leakage: Incorporates multiple timestamps for each sample to prevent data leakage.