Statistically Profiling Biases in Natural Language Reasoning Datasets and Models
원문 초록
Recent work has indicated that many natural language understanding and reasoning datasets contain statistical cues that may be taken advantaged of by NLP models whose capability may thus be grossly overestimated. To discover the potential weakness in the models, some human-designed stress tests have been proposed but they are expensive to create and do not generalize to arbitrary models. We propose a light-weight and general statistical profiling framework, ICQ (I-See-Cue), which automatically identifies possible biases in any multiple-choice NLU datasets without the need to create any additional test cases, and further evaluates through blackbox testing the extent to which models may exploit these biases.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Vision Language Action Models in Robotic Manipulation: A Systematic Review
A Survey on Vision-Language-Action Models for Autonomous Driving
Training-free LLM Merging for Multi-task Learning
Dialect Normalization using Large Language Models and Morphological Rules
STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving
EdgeProfiler: A Fast Profiling Framework for Lightweight LLMs on Edge Using Analytical Model
토론
이 논문과 연결된 포럼 글이 아직 없습니다.