Chain of Logic: Rule-Based Reasoning with Large Language Models
원문 초록
Rule-based reasoning, a fundamental type of legal reasoning, enables us to draw conclusions by accurately applying a rule to a set of facts. We explore causal language models as rule-based reasoners, specifically with respect to compositional rules - rules consisting of multiple elements which form a complex logical expression. Reasoning about compositional rules is challenging because it requires multiple reasoning steps, and attending to the logical relationships between elements. We introduce a new prompting method, Chain of Logic, which elicits rule-based reasoning through decomposition (solving elements as independent threads of logic), and recomposition (recombining these sub-answers to resolve the underlying logical expression). This method was inspired by the IRAC (Issue, Rule, Application, Conclusion) framework, a sequential reasoning approach used by lawyers. We evaluate chain of logic across eight rule-based reasoning tasks involving three distinct compositional rules from the LegalBench benchmark and demonstrate it consistently outperforms other prompting methods, including chain of thought and self-ask, using open-source and commercial language models.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Can ChatGPT Perform Reasoning Using the IRAC Method in Analyzing Legal Scenarios Like a Lawyer?
LLM-based HSE Compliance Assessment: Benchmark, Performance, and Advancements
Incorporating Legal Structure in Retrieval-Augmented Generation: A Case Study on Copyright Fair Use
CaseGen: A Benchmark for Multi-Stage Legal Case Documents Generation
JUREX-4E: Juridical Expert-Annotated Four-Element Knowledge Base for Legal Reasoning
NitiBench: A Comprehensive Studies of LLM Frameworks Capabilities for Thai Legal Question Answering
토론
이 논문과 연결된 포럼 글이 아직 없습니다.