Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 1편

PowerPoint Task Completion (PPTC)

PPTC
PPTC 설명 보기

Recent evaluations of Large Language Models (LLMs) have centered around testing their zero-shot/few-shot capabilities for basic natural language tasks and their ability to translate instructions into tool APIs. However, the evaluation of LLMs utilizing complex tools to finish multi-turn, multi-modal instructions in a complex multi-modal environment has not been investigated. To address this gap, we introduce the PowerPoint Task Completion (PPTC) benchmark to assess LLMs' ability to create and edit PPT files based on user instructions. It contains 279 multi-turn sessions covering diverse topics and hundreds of instructions involving multi-modal operations.