Data used to train machine learning (ML) models can be sensitive. Membership inference attacks (MIAs), attempting to determine whether a particular data record was used to train an ML model, risk violating membership privacy. ML model builders need a principled definition of a metric to quantify the membership privacy risk of (a) individual training data records, (b) computed independently of specific MIAs, (c) which assesses susceptibility to different MIAs, (d) can be used for different applications, and (e) efficiently. None of the prior membership privacy risk metrics simultaneously meet all these requirements. We present SHAPr, a membership privacy metric based on Shapley values which is a leave-one-out (LOO) technique, originally intended to measure the contribution of a training data record on model utility. We conjecture that contribution to model utility can act as a proxy for memorization, and hence represent membership privacy risk. Using ten benchmark datasets, we show that SHAPr is indeed effective in estimating susceptibility of training data records to MIAs. We also show that, unlike prior work, SHAPr is significantly better in estimating susceptibility to newer, and more effective MIA. We apply SHAPr to evaluate the efficacy of several defenses against MIAs: using regularization and removing high risk training data records. Moreover, SHAPr is versatile: it can be used for estimating vulnerability of different subgroups to MIAs, and inherits applications of Shapley values (e.g., data valuation). We show that SHAPr has an acceptable computational cost (compared to naive LOO), varying from a few minutes for the smallest dataset to ~92 minutes for the largest dataset.
In this work, we propose ModelPred, a framework that helps to understand the impact of changes in training data on a trained model. This is critical for building trust in various stages of a machine learning pipeline: from cleaning poor-quality samples and tracking important ones to be collected during data preparation, to calibrating uncertainty of model prediction, to interpreting why certain behaviors of a model emerge during deployment. Specifically, ModelPred learns a parameterized function that takes a dataset $S$ as the input and predicts the model obtained by training on $S$. Our work differs from the recent work of Datamodels [1] as we aim for predicting the trained model parameters directly instead of the trained model behaviors. We demonstrate that a neural network-based set function class is capable of learning the complex relationships between the training data and model parameters. We introduce novel global and local regularization techniques to prevent overfitting and we rigorously characterize the expressive power of neural networks (NN) in approximating the end-to-end training process. Through extensive empirical investigations, we show that ModelPred enables a variety of applications that boost the interpretability and accountability of machine learning (ML), such as data valuation, data selection, memorization quantification, and model calibration.
Data Shapley has recently been proposed as a principled framework to quantify the contribution of individual datum in machine learning. It can effectively identify helpful or harmful data points for a learning algorithm. In this paper, we propose Beta Shapley, which is a substantial generalization of Data Shapley. Beta Shapley arises naturally by relaxing the efficiency axiom of the Shapley value, which is not critical for machine learning settings. Beta Shapley unifies several popular data valuation methods and includes data Shapley as a special case. Moreover, we prove that Beta Shapley has several desirable statistical properties and propose efficient algorithms to estimate it. We demonstrate that Beta Shapley outperforms state-of-the-art data valuation methods on several downstream ML tasks such as: 1) detecting mislabeled training data; 2) learning with subsamples; and 3) identifying points whose addition or removal have the largest positive or negative impact on the model.
Federated learning is an emerging decentralized machine learning scheme that allows multiple data owners to work collaboratively while ensuring data privacy. The success of federated learning depends largely on the participation of data owners. To sustain and encourage data owners' participation, it is crucial to fairly evaluate the quality of the data provided by the data owners and reward them correspondingly. Federated Shapley value, recently proposed by Wang et al. [Federated Learning, 2020], is a measure for data value under the framework of federated learning that satisfies many desired properties for data valuation. However, there are still factors of potential unfairness in the design of federated Shapley value because two data owners with the same local data may not receive the same evaluation. We propose a new measure called completed federated Shapley value to improve the fairness of federated Shapley value. The design depends on completing a matrix consisting of all the possible contributions by different subsets of the data owners. It is shown under mild conditions that this matrix is approximately low-rank by leveraging concepts and tools from optimization. Both theoretical analysis and empirical evaluation verify that the proposed measure does improve fairness in many circumstances.
The Shapley value (SV) and Least core (LC) are classic methods in cooperative game theory for cost/profit sharing problems. Both methods have recently been proposed as a principled solution for data valuation tasks, i.e., quantifying the contribution of individual datum in machine learning. However, both SV and LC suffer computational challenges due to the need for retraining models on combinatorially many data subsets. In this work, we propose to boost the efficiency in computing Shapley value or Least core by learning to estimate the performance of a learning algorithm on unseen data combinations. Theoretically, we derive bounds relating the error in the predicted learning performance to the approximation error in SV and LC. Empirically, we show that the proposed method can significantly improve the accuracy of SV and LC estimation.
High-quality data is critical to train performant Machine Learning (ML) models, highlighting the importance of Data Quality Management (DQM). Existing DQM schemes often cannot satisfactorily improve ML performance because, by design, they are oblivious to downstream ML tasks. Besides, they cannot handle various data quality issues (especially those caused by adversarial attacks) and have limited applications to only certain types of ML models. Recently, data valuation approaches (e.g., based on the Shapley value) have been leveraged to perform DQM; yet, empirical studies have observed that their performance varies considerably based on the underlying data and training process. In this paper, we propose a task-driven, multi-purpose, model-agnostic DQM framework, DataSifter, which is optimized towards a given downstream ML task, capable of effectively removing data points with various defects, and applicable to diverse models. Specifically, we formulate DQM as an optimization problem and devise a scalable algorithm to solve it. Furthermore, we propose a theoretical framework for comparing the worst-case performance of different DQM strategies. Remarkably, our results show that the popular strategy based on the Shapley value may end up choosing the worst data subset in certain practical scenarios. Our evaluation shows that DataSifter achieves and most often significantly improves the state-of-the-art performance over a wide range of DQM tasks, including backdoor, poison, noisy/mislabel data detection, data summarization, and data debiasing.
Valuation problems, such as feature interpretation, data valuation and model valuation for ensembles, become increasingly more important in many machine learning applications. Such problems are commonly solved by well-known game-theoretic criteria, such as Shapley value or Banzhaf value. In this work, we present a novel energy-based treatment for cooperative games, with a theoretical justification by the maximum entropy framework. Surprisingly, by conducting variational inference of the energy-based model, we recover various game-theoretic valuation criteria through conducting one-step fixed point iteration for maximizing the mean-field ELBO objective. This observation also verifies the rationality of existing criteria, as they are all attempting to decouple the correlations among the players through the mean-field approach. By running fixed point iteration for multiple steps, we achieve a trajectory of the valuations, among which we define the valuation with the best conceivable decoupling error as the Variational Index. We prove that under uniform initializations, these variational valuations all satisfy a set of game-theoretic axioms. We experimentally demonstrate that the proposed Variational Index enjoys lower decoupling error and better valuation performance on certain synthetic and real-world valuation problems.
Client contribution evaluation, also known as data valuation, is a crucial approach in federated learning(FL) for client selection and incentive allocation. However, due to restrictions of accessibility of raw data, only limited information such as local weights and local data size of each client is open for quantifying the client contribution. Using data size from available information, we introduce an empirical evaluation method called Federated Client Contribution Evaluation through Accuracy Approximation(FedCCEA). This method builds the Accuracy Approximation Model(AAM), which estimates a simulated test accuracy using inputs of sampled data size and extracts the clients' data quality and data size to measure client contribution. FedCCEA strengthens some advantages: (1) enablement of data size selection to the clients, (2) feasible evaluation time regardless of the number of clients, and (3) precise estimation in non-IID settings. We demonstrate the superiority of FedCCEA compared to previous methods through several experiments: client contribution distribution, client removal, and robustness test to partial participation.
The reliability of machine learning models can be compromised when trained on low quality data. Many large-scale medical imaging datasets contain low quality labels extracted from sources such as medical reports. Moreover, images within a dataset may have heterogeneous quality due to artifacts and biases arising from equipment or measurement errors. Therefore, algorithms that can automatically identify low quality data are highly desired. In this study, we used data Shapley, a data valuation metric, to quantify the value of training data to the performance of a pneumonia detection algorithm in a large chest X-ray dataset. We characterized the effectiveness of data Shapley in identifying low quality versus valuable data for pneumonia detection. We found that removing training data with high Shapley values decreased the pneumonia detection performance, whereas removing data with low Shapley values improved the model performance. Furthermore, there were more mislabeled examples in low Shapley value data and more true pneumonia cases in high Shapley value data. Our results suggest that low Shapley value indicates mislabeled or poor quality images, whereas high Shapley value indicates data that are valuable for pneumonia detection. Our method can serve as a framework for using data Shapley to denoise large-scale medical imaging datasets.
Federated learning (FL) is a popular technique to train machine learning (ML) models on decentralized data sources. In order to sustain long-term participation of data owners, it is important to fairly appraise each data source and compensate data owners for their contribution to the training process. The Shapley value (SV) defines a unique payoff scheme that satisfies many desiderata for a data value notion. It has been increasingly used for valuing training data in centralized learning. However, computing the SV requires exhaustively evaluating the model performance on every subset of data sources, which incurs prohibitive communication cost in the federated setting. Besides, the canonical SV ignores the order of data sources during training, which conflicts with the sequential nature of FL. This paper proposes a variant of the SV amenable to FL, which we call the federated Shapley value. The federated SV preserves the desirable properties of the canonical SV while it can be calculated without incurring extra communication cost and is also able to capture the effect of participation order on data value. We conduct a thorough empirical study of the federated SV on a range of tasks, including noisy label detection, adversarial participant detection, and data summarization on different benchmark datasets, and demonstrate that it can reflect the real utility of data sources for FL and has the potential to enhance system robustness, security, and efficiency. We also report and analyze "failure cases" and hope to stimulate future research.
Shapley value is a classic notion from game theory, historically used to quantify the contributions of individuals within groups, and more recently applied to assign values to data points when training machine learning models. Despite its foundational role, a key limitation of the data Shapley framework is that it only provides valuations for points within a fixed data set. It does not account for statistical aspects of the data and does not give a way to reason about points outside the data set. To address these limitations, we propose a novel framework -- distributional Shapley -- where the value of a point is defined in the context of an underlying data distribution. We prove that distributional Shapley has several desirable statistical properties; for example, the values are stable under perturbations to the data points themselves and to the underlying data distribution. We leverage these properties to develop a new algorithm for estimating values from data, which comes with formal guarantees and runs two orders of magnitude faster than state-of-the-art algorithms for computing the (non-distributional) data Shapley values. We apply distributional Shapley to diverse data sets and demonstrate its utility in a data market setting.
Quantifying the value of data is a fundamental problem in machine learning. Data valuation has multiple important use cases: (1) building insights about the learning task, (2) domain adaptation, (3) corrupted sample discovery, and (4) robust learning. To adaptively learn data values jointly with the target task predictor model, we propose a meta learning framework which we name Data Valuation using Reinforcement Learning (DVRL). We employ a data value estimator (modeled by a deep neural network) to learn how likely each datum is used in training of the predictor model. We train the data value estimator using a reinforcement signal of the reward obtained on a small validation set that reflects performance on the target task. We demonstrate that DVRL yields superior data value estimates compared to alternative methods across different types of datasets and in a diverse set of application scenarios. The corrupted sample discovery performance of DVRL is close to optimal in many regimes (i.e. as if the noisy samples were known apriori), and for domain adaptation and robust learning DVRL significantly outperforms state-of-the-art by 14.6% and 10.8%, respectively.
Given a data set $\mathcal{D}$ containing millions of data points and a data consumer who is willing to pay for \$$X$ to train a machine learning (ML) model over $\mathcal{D}$, how should we distribute this \$$X$ to each data point to reflect its "value"? In this paper, we define the "relative value of data" via the Shapley value, as it uniquely possesses properties with appealing real-world interpretations, such as fairness, rationality and decentralizability. For general, bounded utility functions, the Shapley value is known to be challenging to compute: to get Shapley values for all $N$ data points, it requires $O(2^N)$ model evaluations for exact computation and $O(N\log N)$ for $(\epsilon, \delta)$-approximation. In this paper, we focus on one popular family of ML models relying on $K$-nearest neighbors ($K$NN). The most surprising result is that for unweighted $K$NN classifiers and regressors, the Shapley value of all $N$ data points can be computed, exactly, in $O(N\log N)$ time -- an exponential improvement on computational complexity! Moreover, for $(\epsilon, \delta)$-approximation, we are able to develop an algorithm based on Locality Sensitive Hashing (LSH) with only sublinear complexity $O(N^{h(\epsilon,K)}\log N)$ when $\epsilon$ is not too small and $K$ is not too large. We empirically evaluate our algorithms on up to $10$ million data points and even our exact algorithm is up to three orders of magnitude faster than the baseline approximation algorithm. The LSH-based approximation algorithm can accelerate the value calculation process even further. We then extend our algorithms to other scenarios such as (1) weighed $K$NN classifiers, (2) different data points are clustered by different data curators, and (3) there are data analysts providing computation who also requires proper valuation.
As data becomes the fuel driving technological and economic growth, a fundamental challenge is how to quantify the value of data in algorithmic predictions and decisions. For example, in healthcare and consumer markets, it has been suggested that individuals should be compensated for the data that they generate, but it is not clear what is an equitable valuation for individual data. In this work, we develop a principled framework to address data valuation in the context of supervised machine learning. Given a learning algorithm trained on $n$ data points to produce a predictor, we propose data Shapley as a metric to quantify the value of each training datum to the predictor performance. Data Shapley value uniquely satisfies several natural properties of equitable data valuation. We develop Monte Carlo and gradient-based methods to efficiently estimate data Shapley values in practical settings where complex learning algorithms, including neural networks, are trained on large datasets. In addition to being equitable, extensive experiments across biomedical, image and synthetic data demonstrate that data Shapley has several other benefits: 1) it is more powerful than the popular leave-one-out or leverage score in providing insight on what data is more valuable for a given learning task; 2) low Shapley value data effectively capture outliers and corruptions; 3) high Shapley value data inform what type of new data to acquire to improve the predictor.
"How much is my data worth?" is an increasingly common question posed by organizations and individuals alike. An answer to this question could allow, for instance, fairly distributing profits among multiple data contributors and determining prospective compensation when data breaches happen. In this paper, we study the problem of data valuation by utilizing the Shapley value, a popular notion of value which originated in cooperative game theory. The Shapley value defines a unique payoff scheme that satisfies many desiderata for the notion of data value. However, the Shapley value often requires exponential time to compute. To meet this challenge, we propose a repertoire of efficient algorithms for approximating the Shapley value. We also demonstrate the value of each training instance for various benchmark datasets.
많은 머신러닝(machine learning) 과제에서 양질의 데이터 양은 데이터 소유자가 로컬에서 보유한 것으로 제한된다. 양질의 데이터 집합은 외부 데이터 에이전트(agent)와의 거래나 공유를 통해 확장될 수 있다. 그러나 외부 데이터는 오염되어 있거나 바람직하지 않은 샘플 다양성을 유입시킬 수 있으며, 이는 희귀 질환 진단이나 추천 시스템과 같은 개인화된 머신러닝 과제의 성능을 저하시킬 수 있다. 따라서 데이터 구매자는 데이터 획득 이전에 품질 보증이 필요하다. 기존 연구들은 주로 서로 다른 에이전트의 데이터에 대한 분포 가정에 의존하여, 품질 검사를 비용이 많이 드는 데이터 가치 평가(valuation) 절차를 수반하는 사후(post-hoc) 단계로 미뤄왔다. 본 논문은 소량의 데이터만 검사함으로써 모델 개인화(personalization)에 가장 유용한 데이터를 보유한 외부 데이터 에이전트를 식별하는, 분포에 대한 가정이 없고(distribution-free) 오염을 인지하는 데이터 획득 프레임워크를 제안한다. 이를 위해, 전체 데이터 획득에 앞서 수행되는 새로운 이표본 검정(two-sample testing) 절차를 도입하며, 이는 컨포멀(conformal) 이상치 탐지(outlier detection)에 대한 엄밀한 이론적 토대에 기반하여 어떤 에이전트의 데이터가 오염 임계값을 초과하는지를 판단한다. 제안된 검정은 '컨포멀 데이터 오염 검정(conformal data contamination test)'이라 명명되며, 임의의 오염 수준(arbitrary contamination level)에서도 유효성을 유지하고, 새롭게 제시된 스토리(Storey) 유형 검정은 벤자민-호크버그(Benjamini-Hochberg) 절차를 통해 유한 표본(finite-sample)에서의 거짓 발견율(false discovery rate) 제어를 입증 가능하게 한다. 다양한 협력 학습(collaborative learning) 시나리오에 대한 실증적 평가는 본 접근법의 강건성과 효과성을 보여준다. 전반적으로, 컨포멀 데이터 오염 검정은 통계적으로 엄밀한 품질 보증과 함께 데이터를 집계하는 범용적인 절차로서 차별성을 가진다.
대규모 언어 모델(LLM)의 정렬은 점점 더 데이터 품질에 의해 병목이 발생하고 있다. 데이터셋이 확장됨에 따라, 방대한 선호(preference) 및 인스트럭션 튜닝(instruction tuning) 코퍼스에는 필연적으로 숨겨진 구조적 모순, 안전성(safety) 위험, 체계적인 인간 어노테이션 오류가 누적된다. 의미 중복 제거(semantic deduplication)나 LLM-as-a-judge와 같은 표준 데이터셋 감사 방법은 개별 레코드의 실제 예측적 영향을 포착하기 어렵고, 종종 깊은 기능적 규칙 충돌을 놓친다. 이를 해결하기 위해, 우리는 반복적 모델 재학습 없이 Shapley value를 근사하는 확장 가능하고 추론 전용 데이터 가치 평가 파이프라인을 제안한다. 의미적 k-NN 이웃을 방향 그래프로 매핑함으로써, 우리 프레임워크는 제로샷(zero-shot) 및 원샷(one-shot) 조건부 로그 가능도(log-likelihood) 변화를 사용하여 참조 LLM의 확률 분포를 통해 데이터 유틸리티를 직접 평가한다. 우리 파이프라인은 이러한 예측적 영향 점수를 국소화된 advantage 지표로 변환하여 그래디언트 충돌 레코드를 분리한다. 우리는 두 개의 철저히 검증된 정렬 데이터셋을 정화하는 데 있어 파이프라인의 효과를 입증한다. 첫째, HelpSteer2 데이터셋에 우리 파이프라인을 적용하면 수동 감사 탐색 공간을 99.1% 줄였으며, 다양한 실패 모드에 걸쳐 잘못 라벨링된 레코드를 성공적으로 발견했다. 둘째, Anthropic의 HH-RLHF 학습 및 평가 분할에 자동화된 감사 전략을 적용하면 수천 건의 숨겨진 안전성 및 사실적 선호 역전(preference inversion)을 식별했다. 결정적으로, 이 감사를 평가 분할로 확장함으로써 현재 벤치마크 무결성의 심각한 취약점을 드러낸다: 고성능 모델은 더 안전하거나 더 도움이 되는 응답을 자주 예측하지만, 객관적으로 결함이 있는 인간 정답(ground-truth) 라벨에 의해 페널티를 받는다. 전반적으로, 우리 연구는 인간 라벨 실패를 밝혀내고, 평가 벤치마크를 정화하며, LLM 정렬 데이터의 무결성을 보장하는 수학적으로 근거 있는 고효율 진단 도구를 제공한다.
DPO, IPO 등 Direct Alignment Algorithm(DAA)을 활용한 LLM 정렬 시 여러 선호 데이터셋의 기여도를 Shapley 값으로 평가하려면 지수적 수의 파인튜닝이 필요한 문제를 해결하기 위해, 순차적 선호 최적화(Sequential Preference Optimization)를 도입하여 로그-정책 비율 공간에서의 가법적 합성 구조를 활용해 필요한 파인튜닝 횟수를 데이터 소스 수에 대해 선형으로 줄였다. 이를 통해 실제 선호 데이터셋에 대한 Shapley 값을 효율적으로 계산하고 각 소스가 모델 정렬에 미치는 영향을 정량적으로 규명했다.
검증 세트에 노이즈를 추가하는 소폭의 변화만으로도 샤플리 값(Shapley value)이 일관되게 0으로 수렴하는 방향성 이동이 발생한다는 체계적 취약점을 KNN-Shapley 프레임워크를 통해 규명하였으며, 이는 경계 샘플의 값 범위 축소와 비경계 샘플의 양수 값 감소라는 두 가지 메커니즘에 기인함을 보였다. 이러한 왜곡을 완화하기 위해 엔트로피 인식 정규화 및 경계 인식 검증 전략을 제안하여 머신러닝 마켓플레이스에서의 공정한 데이터 가치 평가를 지원한다.
데이터 가치 평가는 체계적인 데이터 큐레이션, 품질 관리 및 강건한 학습을 가능하게 하기 위해 개별 샘플의 내재적 품질을 정량화합니다. 의료, 금융 및 산업 모니터링과 같은 중요한 도메인의 시계열 데이터의 경우, 효과적인 가치 평가 방법은 필수적이지만 근본적으로 부족합니다. 기존 접근 방식은 모델 의존적이어서 일반화 가능성을 제한하거나, 독립 동일 분포(i.i.d.) 데이터용으로 설계되어 순차 데이터에 내재된 시간적 의존성, 다중 스케일 패턴 및 비정상성 동역학을 포착하지 못합니다. 우리는 평가 대상 데이터와 참조 데이터 간의 분포 차이를 최소화하는 데 대한 시계열 세그먼트의 한계 기여도를 기반으로 가치를 매기는 학습 무관 프레임워크인 TimeLAVA를 소개합니다. 그 핵심은 시간적 국소화를 위한 다중 스케일 웨이블릿 변환과 분포 변화에 대한 강건성을 위한 불균형 최적 전송을 결합한 새로운 선택적 웨이블릿 기반 와서스타인 차이입니다. 세그먼트 값은 모델 학습 없이 민감도 분석을 통해 효율적으로 계산되며 점별 점수로 집계됩니다. 우리는 가치 평가와 모델 무관 일반화를 연결하는 이론적 보장을 제공하고 이상치 오염에 대한 유계 민감도를 증명합니다. 이상 탐지, 데이터 가지치기 및 레이블 노이즈 탐지에 걸친 광범위한 실험을 통해 TimeLAVA가 다양한 실제 데이터셋에서 기존 방법보다 훨씬 더 유익한 가치 점수를 생성함을 입증합니다.