Through the LLM Looking Glass: A Socratic Probing of Donkeys, Elephants, and Markets
원문 초록
While detecting and avoiding bias in LLM-generated text is becoming increasingly important, media bias often remains subtle and subjective, making it particularly difficult to identify and mitigate. In this study, we assess media bias in LLM-generated content and LLMs' ability to detect subtle ideological bias. We conduct this evaluation using two datasets, PoliGen and EconoLex, covering political and economic discourse, respectively. We evaluate seven widely used LLMs by prompting them to generate articles and analyze their ideological preferences via Socratic probing. By using our self-contained Socratic approach, the study aims to directly measure the models' biases rather than relying on external interpretations, thereby minimizing subjective judgments about media bias. Our results reveal a consistent preference of Democratic over Republican positions across all models. Conversely, in economic topics, biases vary among Western LLMs, while those developed in China lean more strongly toward socialism.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles
Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectors
Narrative Shift Detection: A Hybrid Approach of Dynamic Topic Models and Large Language Models
Characterizing Linguistic Shifts in Croatian News via Diachronic Word Embeddings
How Grounded is Wikipedia? A Study on Structured Evidential Support
Intent Factored Generation: Unleashing the Diversity in Your Language Model
토론
이 논문과 연결된 포럼 글이 아직 없습니다.