Rethinking Data Shapley for Data Selection Tasks: Misleads and Merits
원문 초록
Data Shapley provides a principled approach to data valuation and plays a crucial role in data-centric machine learning (ML) research. Data selection is considered a standard application of Data Shapley. However, its data selection performance has shown to be inconsistent across settings in the literature. This study aims to deepen our understanding of this phenomenon. We introduce a hypothesis testing framework and show that Data Shapley's performance can be no better than random selection without specific constraints on utility functions. We identify a class of utility functions, monotonically transformed modular functions, within which Data Shapley optimally selects data. Based on this insight, we propose a heuristic for predicting Data Shapley's effectiveness in data selection tasks. Our experiments corroborate these findings, adding new insights into when Data Shapley may or may not succeed.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Faithful Group Shapley Value
Shapley-Guided Utility Learning for Effective Graph Inference Data Valuation
FW-Shapley: Real-time Estimation of Weighted Shapley Values
ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Party LLM Data Valuation
DUPRE: Data Utility Prediction for Efficient Data Valuation
Data Valuation using Neural Networks for Efficient Instruction Fine-Tuning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.