Could Thinking Multilingually Empower LLM Reasoning?
원문 초록
Previous work indicates that large language models exhibit a significant "English bias", i.e. they often perform better when tasks are presented in English. Interestingly, we have observed that using certain other languages in reasoning tasks can yield better performance than English. However, this phenomenon remains under-explored. In this paper, we explore the upper bound of harnessing multilingualism in reasoning tasks, suggesting that multilingual reasoning promises significantly (by nearly 10 Acc@$k$ points) and robustly (tolerance for variations in translation quality and language choice) higher upper bounds than English-only reasoning. Besides analyzing the reason behind the upper bound and challenges in reaching it, we also find that common answer selection methods cannot achieve this upper bound, due to their limitations and biases. These insights could pave the way for future research aimed at fully harnessing the potential of multilingual reasoning in LLMs.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
FinBERT-QA: Financial Question Answering with pre-trained BERT Language Models
SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models
Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks
HGOT: Hierarchical Graph of Thoughts for Retrieval-Augmented In-Context Learning in Factuality Evaluation
When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards
토론
이 논문과 연결된 포럼 글이 아직 없습니다.