A Study into Investigating Temporal Robustness of LLMs
원문 초록
Large Language Models (LLMs) encapsulate a surprising amount of factual world knowledge. However, their performance on temporal questions and historical knowledge is limited because they often cannot understand temporal scope and orientation or neglect the temporal aspect altogether. In this study, we aim to measure precisely how robust LLMs are for question answering based on their ability to process temporal information and perform tasks requiring temporal reasoning and temporal factual knowledge. Specifically, we design eight time-sensitive robustness tests for factual information to check the sensitivity of six popular LLMs in the zero-shot setting. Overall, we find LLMs lacking temporal robustness, especially to temporal reformulations and the use of different granularities of temporal references. We show how a selection of these eight tests can be used automatically to judge a model's temporal robustness for user questions on the fly. Finally, we apply the findings of this study to improve the temporal QA performance by up to 55 percent.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks
ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
From Roots to Rewards: Dynamic Tree Reasoning with RL
Describe Anything Model for Visual Question Answering on Text-rich Images
Warehouse Spatial Question Answering with LLM Agent
DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge
토론
이 논문과 연결된 포럼 글이 아직 없습니다.