Scalable Source Selection for Machine Learning Tasks

원문 초록

Given a query table, existing data lake discovery techniques seek to identify data sources related to the query table through schema similarity and joinability. The discovered sources, however, are not always useful for downstream analyses; while some improve task generalization, some have the potential to inadvertently introduce harmful noise or distribution shifts. We formalize source selection as the problem of finding a subset of the discovered sources that maximizes downstream task utility. We propose Splice, a target- and cost-aware source selection system that adopts the idea of gene splicing to identify sources which provide additional signals that the union of all sources misses. To speed up the process, we develop two surrogate variants, Splice-DP and Splice-Grad, that estimate the marginal contribution of subsets to task utility by leveraging data profiles in a meta-learning approach and gradient alignment, respectively. On classification tasks over two real-world benchmarks derived from SOTA table union search tools, Splice shows 3-5 percentage points improvement in binary F1 scores compared to the case when all discovered tables are considered. On five additional real-world ML tasks spanning classification and regression, Splice improves task utility over the all-sources baseline by 1.8-13.7 points, while Splice-DP and Splice-Grad match this quality with 3-7x fewer model evaluations (up to 77x at scale).