Evaluating the Accuracy of Chatbots in Financial Literature
원문 초록
We evaluate the reliability of two chatbots, ChatGPT (4o and o1-preview versions), and Gemini Advanced, in providing references on financial literature and employing novel methodologies. Alongside the conventional binary approach commonly used in the literature, we developed a nonbinary approach and a recency measure to assess how hallucination rates vary with how recent a topic is. After analyzing 150 citations, ChatGPT-4o had a hallucination rate of 20.0% (95% CI, 13.6%-26.4%), while the o1-preview had a hallucination rate of 21.3% (95% CI, 14.8%-27.9%). In contrast, Gemini Advanced exhibited higher hallucination rates: 76.7% (95% CI, 69.9%-83.4%). While hallucination rates increased for more recent topics, this trend was not statistically significant for Gemini Advanced. These findings emphasize the importance of verifying chatbot-provided references, particularly in rapidly evolving fields.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Mitigating Object Hallucinations via Sentence-Level Early Intervention
UQLM: A Python Package for Uncertainty Quantification in Large Language Models
Generalized Adaptive Transfer Network: Enhancing Transfer Learning in Reinforcement Learning Across Domains
Mitigating Hallucination of Large Vision-Language Models via Dynamic Logits Calibration
KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality
DiscoSG: Towards Discourse-Level Text Scene Graph Parsing through Iterative Graph Refinement
토론
이 논문과 연결된 포럼 글이 아직 없습니다.