Towards Auditing Large Language Models: Improving Text-based Stereotype Detection
원문 초록
Large Language Models (LLM) have made significant advances in the recent past becoming more mainstream in Artificial Intelligence (AI) enabled human-facing applications. However, LLMs often generate stereotypical output inherited from historical data, amplifying societal biases and raising ethical concerns. This work introduces i) the Multi-Grain Stereotype Dataset, which includes 52,751 instances of gender, race, profession and religion stereotypic text and ii) a novel stereotype classifier for English text. We design several experiments to rigorously test the proposed model trained on the novel dataset. Our experiments show that training the model in a multi-class setting can outperform the one-vs-all binary counterpart. Consistent feature importance signals from different eXplainable AI tools demonstrate that the new model exploits relevant text features. We utilise the newly created model to assess the stereotypic behaviour of the popular GPT family of models and observe the reduction of bias over time. In summary, our work establishes a robust and practical framework for auditing and evaluating the stereotypic bias in LLM.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Enhancing interpretability of rule-based classifiers through feature graphs
Profiling Electric Vehicles via Early Charging Voltage Patterns
The Reader is the Metric: How Textual Features and Reader Profiles Explain Conflicting Evaluations of AI Creative Writing
Multi-criteria Rank-based Aggregation for Explainable AI
EPIC: Explanation of Pretrained Image Classification Networks via Prototype
FairSHAP: Preprocessing for Fairness Through Attribution-Based Data Augmentation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.