Exploring the psychology of LLMs' Moral and Legal Reasoning
원문 초록
Large language models (LLMs) exhibit expert-level performance in tasks across a wide range of different domains. Ethical issues raised by LLMs and the need to align future versions makes it important to know how state of the art models reason about moral and legal issues. In this paper, we employ the methods of experimental psychology to probe into this question. We replicate eight studies from the experimental literature with instances of Google's Gemini Pro, Anthropic's Claude 2.1, OpenAI's GPT-4, and Meta's Llama 2 Chat 70b. We find that alignment with human responses shifts from one experiment to another, and that models differ amongst themselves as to their overall alignment, with GPT-4 taking a clear lead over all other models we tested. Nonetheless, even when LLM-generated responses are highly correlated to human responses, there are still systematic differences, with a tendency for models to exaggerate effects that are present among humans, in part by reducing variance. This recommends caution with regards to proposals of replacing human participants with current state-of-the-art LLMs in psychological research and highlights the need for further research about the distinctive aspects of machine psychology.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Can ChatGPT Perform Reasoning Using the IRAC Method in Analyzing Legal Scenarios Like a Lawyer?
LLM-based HSE Compliance Assessment: Benchmark, Performance, and Advancements
Incorporating Legal Structure in Retrieval-Augmented Generation: A Case Study on Copyright Fair Use
CaseGen: A Benchmark for Multi-Stage Legal Case Documents Generation
JUREX-4E: Juridical Expert-Annotated Four-Element Knowledge Base for Legal Reasoning
NitiBench: A Comprehensive Studies of LLM Frameworks Capabilities for Thai Legal Question Answering
토론
이 논문과 연결된 포럼 글이 아직 없습니다.