arXiv:2602.06054v4 cs.CL

Novelty Reviewer: A Large-Scale Dataset and Model for Reviewer-Aligned Novelty Assessment

Abeer Mostafa, Thi Huyen Nguyen, Zahra Ahmadi

제출 2026-11-04

원문 초록

Assessing novelty in AI research is arguably the most consequential yet least reliable step in peer review. Reviewer judgments of novelty remain inconsistent and dependent on comparisons to prior work that are often incomplete. We release a large-scale Novelty Reviewer Dataset built from 102,021 official reviews of 37,899 ICLR and NeurIPS submissions (2022-2025). For each submission, it provides the reviewers' novelty statements, a paper-level novelty assessment, and a novelty score on a four-point scale. On top of this dataset, we build Novelty Reviewer, a literature-aware framework that learns how reviewers judge novelty and grounds these judgments in semantic comparisons to existing research. We fine-tune a large language model on the dataset to capture reviewer-aligned novelty evaluation behavior. Given a manuscript, the framework extracts structured representations of its ideas, methods, and claims; retrieves semantically related papers; and constructs a knowledge graph that enables concept-level comparison to prior work. Based on the retrieved evidence, the model produces calibrated novelty scores alongside human-like explanatory assessments, reducing overestimation and improving consistency relative to existing approaches. Furthermore, we analyze different LLMs' abilities to assess originality and detect potential conceptual plagiarism. We publicly release the dataset, the model, and the code with documentation to support research on automated novelty assessment and peer review.

제출 이력

확인된 버전만 표시합니다. 전체 제출 이력은 arXiv 원문에서 확인하세요.

  1. arXiv:2602.06054v4제출 시각 미확인

함께 살펴볼 논문

공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.

공통 태그로 연결된 공개 논문이 아직 없습니다.

토론

이 논문과 연결된 포럼 글이 아직 없습니다.