Says Who? Effective Zero-Shot Annotation of Focalization
원문 초록
Focalization, the perspective through which narrative is presented, is encoded via a wide range of lexico-grammatical features and is subject to reader interpretation. Even trained annotators frequently disagree on correct labels, suggesting this task is both qualitatively and computationally challenging. In this work, we test how well five contemporary large language model (LLM) families and two baselines perform when annotating short literary excerpts for focalization. Despite the challenging nature of the task, we find that LLMs show comparable performance to trained human annotators, with GPT-4o achieving an average F1 of 84.79%. Further, we demonstrate that the log probabilities output by GPT-family models frequently reflect the difficulty of annotating particular excerpts. Finally, we provide a case study analyzing sixteen Stephen King novels, demonstrating the usefulness of this approach for computational literary studies and the insights gleaned from examining focalization at scale.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Specification-based Security Evaluation of Real-World Agents Tool-Calling under Long-Horizon Context Tasks
Assay2Mol: large language model-based drug design using BioAssay context
Open Source Planning & Control System with Language Agents for Autonomous Scientific Discovery
Detecting Referring Expressions in Visually Grounded Dialogue with Autoregressive Language Models
mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scale
AgentStealth: Reinforcing Large Language Model for Anonymizing User-generated Text
토론
이 논문과 연결된 포럼 글이 아직 없습니다.